Out of a sense of "noblesse oblige", he became a philanthropist and entered politics, first as a Democrat, but then as one of the organizers of the Free Soil Party, which joined the Republican Party in 1856. In the 1860 presidential election, he was a presidential elector for Abraham Lincoln and Hannibal Hamiln. In 1861, he was a member of the Washington peace conference, an unofficial gathering of Northern and Southern moderates attempted to avert war. But after war became inevitable, he considered it his duty to volunteer.