Simon was born on July 4, 1927 in The Bronx, New York City to a Jewish family. He studied at New York University and at the University of Denver. Simon was married five times. The first four ended in divorce. He was married to Elaine Joyce from 1999 until his death in 2018. Simon had three children, including one adopted child. 