Smith began his rugby career in 1895 playing rugby union for the City Rugby Club in Auckland. He first represented his home province Auckland in 1896 and, in the following year made, his début for the New Zealand national team against New South Wales. However, in the following years, Smith played little rugby, instead preferring to concentrate on track. He made a comeback in 1901, gaining All Black selection, before disappearing again until he was enticed back to the game with the prospect of joining the 'Originals' tour to the British Isles and France in 1905.