He was born at Murrumburrah near Yass, New South Wales. His parents were Michael Dunn and Margaret Kelly. Dunn was only a small youth and worked as a jockey. He had robbed a Chinese gold miner and been in trouble with the police. 