In 1947, Carey married Helen Owen (1924–1974). The two had a very happy marriage, and they became the parents of Alexandria, Christopher, Susan, Peter, Hugh, Jr., Michael, Donald, Marianne, Nancy, Helen, Bryan, Paul, Kevin, and Thomas. Beginning in 1961, the family resided at 61 Prospect Park West, a 1910 Park Slope mansion built for the daughter of Bon Ami Company chairman and Progressive Era philanthropist William H. Childs; a decade later, Carey sold the home to journalist Pete Hamill. His wife, Helen, died of breast cancer in 1974. Peter and Hugh, Jr. died in an automobile accident in 1969. Carey was devastated by the death of his wife and laid to rest any plans for the White House. Paul, who served as White House Special Assistant to President Bill Clinton as well as 77th Commissioner of the Securities and Exchange Commission, died of cancer in 2001.