Phillips had previously been married to Sarah Gilman, wealthy widow of Phillips' cousin, merchant Nathaniel Gilman, whose large fortune, bequeathed to Phillips, enabled him to endow the academy. The Gilman family also donated to the academy much of the land on which it stands, including the initial 1793 grant by New Hampshire Governor John Taylor Gilman of the Yard, the oldest part of campus; the academy's first class in 1783 included seven Gilmans. In 1814, Nicholas Gilman, signer of the U.S. Constitution, left $1,000 to Exeter to teach "sacred music."