Opposition within his own congregation, spurred by Rankin's attempts to expel slave-owners from the church, finally led him to resign in 1846 after 24 years as minister of the Ripley Presbyterian Church. Over one-third of the church's members left with him and helped Rankin establish what eventually came to be the Free Presbyterian Church, which may have had as many as 72 congregations before the coming of the Civil War. After the war, Rankin welcomed the reunion of the Presbyterian churches in Ripley.