The congregation was established in 1813 in what was then a small riverside settlement known as Fishkill Landing. A small white frame church was built on the present site. The cemetery was also dedicated and received its first body at the same time. Among its earlier notable burials was William Few, Jr., a Founding Father of the United States, who was living in Fishkill at the time of his death. His body was later reinterred at St. Paul's Church in his native Augusta, Georgia.