Mount was born November 26, 1807, in the village of Setauket, New York, on the north shore of eastern Long Island to parents Julia Ann Hawkins (1782–1841) and Thomas Shephard Mount (1778–1814). Mount's parents operated a farm, as well as a store and tavern that bordered the village green in Setauket. Thomas and Julia had eight children. Five survived childbirth: Henry Smith Mount (1802–1841), Shepard Alonzo Mount (1804–1868), Robert Nelson Mount (1806–1883), William Sidney Mount (1807–1868), and Ruth Hawkins Mount (1808–1888).