In 1999 Kelly formed the band Professor Ratbaggy with Hadley (bass guitar, backing vocals), Haymes (keyboards, organ, backing vocals) and Luscombe (drums). Kelly provided guitars and vocals for their debut album, "Professor Ratbaggy", on EMI Records. Songs were written jointly by all group members and their work was a more groove-oriented style compared to Kelly's usual folk or rock formula, using samples, synthesiser and percussion. Kelly's second anthology of lyrics entitled "Don't Start Me Talking" was first published in 1999, with subsequent songs appended in the 2004 edition. This second edition was added to the Victorian Certificate of Education English reading list for Year 12 (final year of secondary schooling) in 2006.