John Frelinghuysen died in 1754, and at the age of twenty, Hardenbergh married Frelinghuysen's widow, Dinah Van Bergh (c.1723–1807) on 18 March 1756 at Raritan, New Jersey. She had two children with Frelinghuysen before his death. Born in Amsterdam, Dinah was the daughter of Louis Van Bergh, a wealthy merchant who was "engaged in East India trade." Hardenbergh proposed as Dinah was strongly considering returning to the Netherlands with her two children. Her diary, dating from February 1746 to late 1747, is held by Special Collections and University Archives, at the Archibald S. Alexander Library of Rutgers University. Hardenbergh and his wife had nine children: Maria (born 1757–1789), Laura (born 1757–1785), Johannes (1759–1798), Elinor (born 1760), Dina (born 1762), Jacob Rutsen (1763–1764), Rachel (born 1765–1845), Jacob Rutsen (born 1768), and Lewis (1771–1810). Hardenbergh's step-son, Frederick Frelinghuysen (1753–1804), served in the Provincial Congress of New Jersey, state legislature, and was elected a United States Senator. He also served as a military officer in the Revolution and in the Whiskey Rebellion (1794) when he attained the rank of major general.