Born in Livingston, West Lothian, Walker had originally harboured an ambition to become a Church of Scotland minister, however his early footballing skills, which saw him recognised by Scotland at schoolboy level, ensured he was destined for a career on the pitch rather than in the pulpit. He played with local sides Berryburn Rangers, Livingston Violet and Broxburn Rangers before joining the Hearts ground staff aged 16 in February 1932. As Scottish clubs could not then officially sign players until the age of 17, Walker played junior football for Linlithgow Rose until his birthday in May.