Isaac Blackford was born on November 6, 1786, in Bound Brook, Somerset County, New Jersey, the son of Joseph and Mary Straats Blackford. Joseph was a merchant, and Mary was the daughter of a farmer. She had been a nurse during the American Revolutionary War and aided American soldiers as a during the battles of Trenton and Princeton, and continued to care for them when smallpox broke out among the army during the winter of 1780–1781. Both of Blackford's parents were Presbyterian, and he remained a member of the church his entire life.