Baily first joined Tottenham Hotspur as a 14-year-old in 1938, and found success at the club's youth set-up. He played cricket for the Essex County Cricket Club Second XI and found work at a printing company and a stock brokerage firm. During World War II he served with the Royal Scots Fusiliers, and saw service in Belgium, the Netherlands and Germany. After the war he signed with Chelsea, but quickly left Stamford Bridge for Tottenham Hotspur after a chance encounter with Jimmy Anderson.