After the American Civil War, Brown joined the Republican Party for a time, and was appointed as chief justice of the Supreme Court of Georgia from 1865 to 1870. Later he rejoined the Democrats, became president of the Western and Atlantic Railroad and began to amass great wealth; he was estimated to be a millionaire by 1880. He benefited from using convicts leased from state, county and local governments in his coal mining operations in Dade County. His Dade Coal Company bought other coal and iron companies, and by 1889 was known as the Georgia Mining, Manufacturing and Investment Company. Finally, he was elected by the state legislature as a two-term U.S. Senator, serving from 1880 to 1891. During this time he was part of the Bourbon Triumvirate alongside fellow prominent Georgia politicians John Brown Gordon and Alfred H. Colquitt.