On March 2, 1870 he was appointed the postmaster of Augusta, Georgia. Senator Hannibal Hamlin of Maine was one of the leading figures behind his appointment. He was subsequently reappointed to that same position in 1874 and again in 1878. His reappointment in 1878 was opposed by most of the Georgia delegation to Congress with the exception of Representative Alexander H. Stephens. Critics labelled Prince a carpetbagger. During that time he also served as a delegate to the Republican National Conventions in 1872, 1876, and 1880. While in Georgia he became a friend of former Vice President of the Confederate States of America Alexander H. Stephens despite their political differences.