Nichols' first marriage was to Patricia Scot; they were married from 1957 to 1960. His second was to Margot Callas, a former "muse" of the poet Robert Graves, from 1963 to 1974. The couple had a daughter together, Daisy Nichols. His third marriage, to Annabel Davis-Goff, produced two children, Max Nichols and Jenny Nichols; they married in 1975 and divorced in 1986. His fourth was to former "ABC World News" anchor Diane Sawyer, whom he married on April 29, 1988. None of his wives was Jewish and his children were not brought up according to a religion, but they identify as Jewish. His son Max married ESPN journalist Rachel Nichols.