Ellen was well educated, having attended a girls seminary in Cortland and music school in the state capital of Albany. She is reported to have owned the first piano in Pompey. She is best known for a notable diary she kept from 1850 to 1858, detailing her life as a wife and mother of a large family. The Wheaton family privately published "The Diary of Ellen Birdseye Wheaton" in 1923. Selections from the diary were reprinted in the Syracuse "Post-Standard" newspaper in March 2002. Birdseye Wheaton died in 1858 at age 42 from tuberculosis.