Carrollton was laid out in 1792, and it was known as Port William initially. It served as the county seat of Gallatin County until 1838 when the county was split, creating Carroll County. Port William was renamed Carrollton after Declaration of Independence signer Charles Carroll and became the seat of the new county. The town's first newspaper, the "Carrollton Eagle", was published in 1848. On September 3, 1863, during the Civil War, Confederates robbed the Southern Bank of Kentucky and were followed by the Union Army. Confederates also stole the sword of prominent resident William Orlando Butler, and he later caught up with the troops and retrieved it. The Louisville & Nashville Railroad was built near town in 1868 and eventually became more important to the town's economy than river traffic. It did cause a spike in agricultural production, especially tobacco.