After the 1951 season, Hirsch finished second behind Otto Graham in voting conducted by the United Press (UP) for the NFL Player of the Year award. He was also selected as a first-team All-Pro player by both the Associated Press (AP) and the UP. He was also selected to play in the Pro Bowl each year from 1951 to 1953. Hirsch had another strong season in 1953, leading the NFL with a career-high average of 23.6 yards per reception. He also finished second in the NFL with 941 receiving yards in 1953 and was selected as a first-team All-Pro by the AP and a second-team All-Pro by the UP.