His scoring feats earned the attention of Tottenham Hotspur, who signed him from Norwich in October 1954. Despite an impressive strike-rate of 15 goals in 32 games Gavin did not settle at White Hart Lane and returned to Norwich in November 1955 as part of the deal that took future England centre-half Maurice Norman to Tottenham from Norwich.
Gavin's outstanding goal-scoring performance attracted the interest of Tottenham Hotspur, who purchased him from Norwich in October 1954. Nevertheless, despite his impressive strike rate of 15 goals in 32 games, Gavin was unable to find a lasting home at White Hart Lane and eventually returned to Norwich in November 1955 as part of a trade that sent Maurice Norman, a future England centre-half, to Tottenham from Norwich.