He was the youngest son of Deacon Nathaniel Ely (1716–1799) and Abigail (Colton) Ely (1724–1770); his mother died when he was 5.
He was the youngest child of Deacon Nathaniel Ely (1716–1799) and Abigail (Colton) Ely (1724–1770); his mother died when he was 5, highlighting a common familial experience.