Wallace was born George Corely Wallace on August 25, 1919 in Clio, Alabama. He studied at University of Alabama. He grew up in a lower middle class family. As a young man, he was an amateur boxer, where he fought as a bantamweight.