Stebbins was born in Lawrence, New York, the youngest of three children. His parents were George Ledyard Stebbins, a wealthy real estate financier who developed Seal Harbor, Maine and helped to establish Acadia National Park, and Edith Alden Candler Stebbins; both parents were native New Yorkers and Episcopalians. Stebbins was known throughout his life as Ledyard, to distinguish himself from his father. The family encouraged their sons' interest in natural history during their periodic journeys to Seal Harbor. In 1914, Edith contracted tuberculosis and the Stebbins moved to Santa Barbara, California to improve her health. In California, Stebbins was enrolled at the Cate School in Carpinteria where he became influenced by Ralph Hoffmann, an American natural history instructor and amateur ornithologist and botanist. After graduating from high school, he embarked on a major in political studies at Harvard. By the third year of his undergraduate study, he had decided to major in botany.