Lou spent a good portion of the early 1950s at Stillman's Boxing Gym. Stillman's Gym was known for two things: its legendary fame as a filthy gym where windows were kept closed so the smell of sweat could not escape; and the large number of celebrities from Hollywood and the boxing world the gym attracted. It was here Lou began friendships with Frank Sinatra, Sammy Davis Jr., Frankie Valli, and other celebrities. Duva, already enamored with the world of boxing, felt his passion for the sport increase every time he visited Stillman's Gym.