As a member of the National Yacht Club, Ballard became an avid racer of Sea Fleas, small outboard hydroplanes. He competed in several regattas, and won the Toronto-Oakville marathon in 1929. Ballard was elected to the Yacht Club's executive committee in January 1930. He participated in the 133-mile Albany, New York-New York City marathon in April 1930, finishing second in his class. About a month later, Ballard and two friends from the Yacht club were hurled from a boat into a frigid Lake Ontario. Ballard was pulled from the water unconscious, but one of his friends died. None of the three was wearing a life jacket.