Barry was married four times. His first three marriages, to Barbara Pickard (1959–63), Jane Birkin (1965–68), and Jane Sidey (1969–78) all ended in divorce. He was married to Laurie from 1978 until his death. The couple had a son, Jonpatrick. Barry had three daughters, Suzanne (Susie) with his first wife, Barbara, Kate with his second wife, Jane, and Sian from a relationship with Ulla Larson between the first two marriages.