Albert G. Porter was born on April 20, 1824, in Lawrenceburg, Indiana, the son of Thomas and Myra Tousey Porter. His father, a veteran of the War of 1812, was a bank teller at the Farmer and Mechanic's Bank. Established by the state government in 1818, the bank folded after the Panic of 1819. His father then served several years as a county clerk until the family moved to Boone County, Kentucky, after his mother inherited her father's farm. The large farm neighbored the Ohio River at the area's most convenient crossing point, and Porter began managing his father's ferryboat, crossing wagons and passengers across the river.