As a swimmer, Handy introduced a number of new swimming strokes to Americans, such as the Australian crawl. He would often wake up early and devise new strokes to give him an edge over other swimmers. Swimming led to him getting a bronze in the 1904 Olympics at St. Louis, Missouri. Twenty years later he was part of the Illinois Athletic Club water polo team at the 1924 Olympics in Paris, France. He broke the record of longest period of time between first and last competition. The team won the bronze at that Olympics. Handy swam almost every day until the last few months of his life.