In the early 1950s, Cohen met the woman he would later marry, Elaine Lustig (née Firstenberg). She and her husband, the graphic designer Alvin Lustig, had moved to New York in 1951. During this time Alvin taught at Yale University and designed covers for New Directions. Following Alvin's death in 1955, Cohen, a close friend of the couple, "suggested that design the book covers Alvin had originally been hired to produce" for Meridian. It was through this working relationship that they became more involved. In 1956, Elaine and Cohen married; they remained married until his death in 1986. The couple lived in Manhattan during these years.