Duva's 23-year-old brother, Carl Duva, introduced young Lou to boxing when the boy was only 10 years old. Lou polished his own boxing skills and by age 12 was both an amateur and ballroom brawler. However Lou as a boxer did not have much luck, although that might have been due to the fact he barely had time to train, having to go out to the street and perform many types of jobs to try to help the Duva family make ends meet.