His second term as governor was undistinguished, but he split with fellow Whig Daniel Webster over a variety of issues, and lost the 1843 election to Democrat Marcus Morton. He was reelected to the Senate in 1845, where he served until 1851. He opposed the Mexican–American War, and worked to prevent the extension of slavery to the territories, although he did not take a hard line on the matter, voting for most of the provisions of the Compromise of 1850. He retired from public service in 1853, and died the next year.