Baldwin was born on August 2, 1924 in Harlem, Manhattan, New York City. He studied at DeWitt Clinton High School and at The New School. Baldwin was gay. He married and had no children. Baldwin died on December 1, 1987 in Saint-Paul de Vence, France from esophageal cancer, aged 63.