After leaving school in 1870, Joe had a number of jobs including driving a delivery cart for a tannery, putting up fences, looking for gold, wood cutting, and shearing. He often visited the Chinese village at Beechworth and became addicted to opium. During the celebrations for the Prince of Wales birthday in 1873, Joe saw a suit of Chinese armour in the Beechworth museum. This armour provided the ideas for the armour the Kelly Gang wore at Glenrowan.