During the 1943–1944 academic year, Hirsch also won varsity letters in basketball (as a center), track (as a broad jumper), and baseball (as a pitcher), becoming the first Michigan athlete to letter in four sports in a single year. He averaged 7.3 points per game for the 1943–44 Michigan Wolverines men's basketball team, compiled a 6-0 record as a pitcher for the Michigan baseball team, placed third in the long jump in the 1944 indoor championship, and led all three teams to Big Ten Conference championships. On May 13, 1944, Hirsch starred in two sports in the same day, winning the broad jump with a distance of 24 feet, 2-1/4 inches at a track meet in Ann Arbor, Michigan, and then traveling to Columbus, Ohio, where he pitched a one-hitter to give Michigan's baseball team a 5-0 victory over Ohio State.