Sara Payson Willis was born in Portland, Maine, to newspaper owner Nathaniel Willis and Hannah Parker. She was the fifth of her nine children. Her older brother Nathaniel Parker Willis became a notable journalist and magazine owner. Her younger brother Richard Storrs Willis became a musician and music journalist, known for writing the melody for "It Came Upon the Midnight Clear". Her other siblings were Lucy Douglas (born 1804), Louisa Harris (1807), Julia Dean (1809), Mary Perry (1813), Edward Payson (1816), and Ellen Holmes Willis (1821).