In 2003, she released "The Grotto", an acoustic solo album of song sketches with personal lyrics set in Providence, Rhode Island, with Andrew Bird on violin and Howe Gelb on piano. On the same date a self-titled album by Throwing Muses was also released, the first since "Limbo". Both were recorded at Steve Rizzo's studio in Rhode Island. When Narcizo was unable to tour on a full-time basis due to other commitments, Hersh formed her power rock trio 50FootWave. Her touring appearances and recording efforts in 2004 and 2005 centered around both 50 Foot Wave and her solo career.