Seeger was born Peter Seeger on May 3, 1919 in Manhattan, New York City. His parents were Protestants. His parents enrolled him at Avon Old Farms Boarding School in Avon, Connecticut. Seeger studied for a short while at Harvard University. He soon dropped out. In 1943, Seeger married his wife Toshi Aline.