He was early attracted to the study of the ecclesiastical history of New England and was frequently called upon to deliver commemorative addresses, some of which were published in book and pamphlet form. Of these, his "Thirteen Historical Discourses" (1839), dealing with the history of New Haven, and his "Four Commemorative Discourses" (1866) may be especially mentioned. The most important of his historical works, however, is his "Genesis of the New England Churches" (1874). He published "A Manual for Young Church Members" (1833); edited, with a biography, the "Select Practical Writings of Richard Baxter" (1831); and was the author of a number of hymns, the best-known of which is the one beginning, "O God, beneath Thy guiding hand Our exiled fathers crossed the sea."