Rose was born and raised in Victoria. He was born at Jacksons Track, and lived in the town of Warragul. Rose had a difficult childhood. He learned to box from his father, Roy. Roy was a skilled fighter who competed in local house shows. At the age of 10, Rose was given a pair of gloves by his teacher, Ian Hawkins. When he was about 15, he began to be taught by Frank Oakes, a trainer in Warragul. Rose later married Frank's daughter, Jenny Oakes.