In the summer of 1994 Pete Finnemore left the band. Neil and Simon played several shows as a two piece, with the bass guitar on a backing track, and Simon as the sole guitarist. Kyle Whipp (formerly of Soul Inside) joined the band as their new bass player and they played their first show with this new line up in 1995 at London's Camden Underworld. The EP "Celebration Drug" was released later on in the year and included the popular track "Whiskey a-go-go". In August of the same year, the band went on a tour of Scandinavia. As well as this, the band released another EP titled "Celibacy and Anadin" which included remixes by Rosetta Stone. After the release of this EP, the band ended their contract with Cleopatra.