In 1860, Swann left the American Party, which dissolved, and joined the merged war-time Union Party. In 1864, he was unanimously nominated as the 33rd Governor of Maryland during its nomination convention. He won election with lieutenant-governor running mate Christopher C. Cox by over 9,000 votes. The only governor elected under the Maryland Constitution of 1864, Swann took the oath of office on January 11, 1865, but did not enter into the duties of the office until one year later (under that constitution, the governor chosen in the November 1864 election could not assume the office until the completion of the term of his predecessor, Augustus W. Bradford, in January 1866); he served until January 1869. In his inaugural address, he encouraged reunion in the State following the American Civil War, and voiced his opposition to slavery, deeming it "a stumbling block in the way of advancement".