From 1945 to 1947, Lee lived in Manhattan. He married Joan Clayton Boocock on December 5, 1947, and in 1949, the couple bought a house on Long Island, living there through 1952. Their daughter Joan Celia "J. C." Lee was born in 1950. Another child, daughter Jan Lee, died three days after delivery in 1953.