He was the youngest son of Deacon Nathaniel Ely (1716–1799) and Abigail (Colton) Ely (1724–1770); his mother died when he was 5.
This individual was born to Deacon Nathaniel Ely and Abigail Colton; his mother's death at the age of five underscores the impact of family loss on young children.