All of the children attended the nearby George Street State School and one by one the brothers joined the Fitzroy Cricket Club as they reached their early teens. The club had a program whereby they gave a medal for every local school to award to the best cricketer in their ranks in that year. The successful student would then be given access to all of the club's facilities. All of the Harvey brothers were recipients were of this medal. At Fitzroy, they came under the influence of former Victorian all-rounder Arthur Liddicut and the club's veteran captain Joe Plant. The Harveys had no formal coaching, and their father, although a regular presence at the club, chose to stay in the background as their respective careers developed. Horace did not advise his sons on how to bat, allowing them to formulate their own style and technique. According to the Harvey brothers, it was their mother who was vocal and extroverted, in contrast to their reserved father. The boys who failed to score runs were given kitchen duty and according to them, their parents never showed favouritism.