Freeman was ordained as a minister by the New York Presbytery in 1840. In 1841, he and his wife moved to Portland, Maine, where he was installed as the first pastor of the Abyssinian Congregational Church, now the third oldest surviving black church in the United States. It became a center of the anti-slavery movement and part of the Underground Railroad. Freeman became the most well-known African American in Maine because of his leadership there; he was a strong supporter of education and also ran a school affiliated with the church. For a decade, he and his wife provided support to fugitive slaves at the church and in their home.