Jacob Rutsen Hardenbergh was born on 22 February 1735/36 at Rosendale near Kingston, Ulster County, New York in the Hudson River Valley. He was the son of Colonel Johannes Hardenbergh, Jr. (1706–1786) and Maria DuBois (1705/06–1790). His parents had married in Kingston on 6 December 1728 and seven children were born to the marriage of which Jacob was the fourth. His father, later served with distinction as a field officer under Washington in the Continental Army, served in New York's Colonial Assembly. Frelinghuysen instructed Hardenbergh from his home, preserved as the Old Dutch Parsonage in Somerville, New Jersey.