The Saints' first release after Kuepper's departure was the live EP, "Paralytic Tonight, Dublin Tomorrow", in March 1980 on Lost Records with Bailey producing. It was followed by a studio album, "The Monkey Puzzle", co-produced by Bailey and Gerry Nixon for Mushroom Records in February 1981. It reached the Top 100 on the Australian Albums Chart. They had shifted to a more melodic pop-rock sound and included Hay on keyboards in the line-up. Hay left again before the next album was released in Australia in 1982 as "I Thought This Was Love, But This Ain't Casablanca" on Mushroom Records and elsewhere as "Out in the Jungle... Where Things Ain't So Pleasant" on New Rose Records. Production was credited to Ricardo Mentalban, and with Bailey, in the Saints, were Hall on bass guitar and Iain Shedden (ex-Jolt) on drums. Additional musicians included Roger Crankwell on saxophone and clarinet, Denis Haines on piano, Paul Neiman on trombone, Steve Sidwell on trumpet and Jess Sutcliffe on piano. The Damned's Brian James guested on lead guitar.