Kemp began her singing career performing live with Steeleye Span while still in her teenage years. This led to her being invited to contribute vocals to several albums by various folk artists. In 1996 she sang with The Carnival Band on their Carols at Christmas CD. In 2002 Park Records released the a cappella folk album Bib and Tuck by 'Maddy Prior And The Girls', a trio composed of Maddy Prior, Rose Kemp and Abbie Lathe. Kemp wrote, and sang lead vocals on, several of the album's tracks. Her first single, "The Milkman Makes Eggs" was an airplay favorite.