Segal was married to Marion Segal Freed from 1956 until they divorced in 1981. He later married Linda Rogoff from 1983 until her death in 1996. He was later married to Sonia Schultz Greenbaum 1996 until his death. He had two children with Freed.