Phelps began his political career serving in the Vermont State House from 1821 to 1832. He was a judge on the Vermont Supreme Court from 1832 to 1838, succeeding to the position left vacant by the death of John C. Thompson. In 1839 he was elected as a Whig to the United States Senate, and he served until 1851. He chaired several committees while serving in the Senate, including the Committee on the Militia, Committee on Revolutionary Claims, Committee on Pensions, Committee on Patents and the Patent Office and the Committee on Territories.