Williams was born on 3 September 1893 at Woollahra, a suburb of Sydney, the third child of Owen Williams, a Victorian-born plumber, and his Scottish wife Isabella, née Wylie. Leaving Woollahra Superior Public School at 14, Harold worked as a messenger-boy, then as a railway stores clerk. He sang with the Waverley Methodist Church choir as a boy soprano and later an amateur baritone; but he found that 'football and cricket were the most absorbing affairs of my life'. He played for Waverley Cricket Club (1906–15) and Rugby Union as a wing-three-quarter with the Eastern Suburbs team, representing New South Wales against New Zealand in August 1914.