While building his career, Burns worked at a Liverpool record shop, Probe Records, which became a meeting place for local musicians. Burns was notorious for his maltreatment of customers, sometimes throwing their purchases at them because he disapproved of their selection. Burns first performed as a member of the short-lived punk band Mystery Girls, which included Pete Wylie and Julian Cope, who gave only one performance, supporting Sham 69 at Eric's Club in November 1977. Cope stated that Burns's performing style drew on that of the transgender punk performer Wayne County.