His mother was descended from the Van Rensselaer family as her mother, Anna Van Rensselaer, was a daughter of Hendrick van Rensselaer (1667–1740), who had married Peter Douw. In addition, his first cousin, Leonard Gansevoort (1754–1834), an Albany lawyer and alderman, was married to Maria Van Rensselaer (1760–1841), the daughter of Col. Kiliaen van Rensselaer (1717–1781), the granddaughter of Hendrick van Rensselaer and the sister of Henry K. Van Rensselaer (1744–1816), Philip Kiliaen van Rensselaer (1747–1798), and Killian K. Van Rensselaer (1763–1845).