He was taught by the Christian Brethren until he was 12 years old. After this, he was sent to work on agricultural stations. He worked at a number of stations during his teenage years, and was often aggravated at being treated differently to White workers. After this, he worked several jobs as a labourer, including on the railway. In his twenties, Barton moved to Adelaide where he worked digging trenches for the South Australian Gas Company. While there, he became an alcoholic, got tuberculosis and spent a year in a sanitorium. By the time he was in his thirties, his addiction to alcohol was so bad that a doctor gave him six months to live. He immediately gave up drinking, and later began working for an Aboriginal alcohol-rehabilitation service in Port Augusta.