While still performing under the name Little Joe Gould, the band headed out on an 11-date tour stretching from California to Louisiana with the Chicago-based band Volta Do Mar. On this tour the band honed the songs for their debut album, which was recorded in several Chicago studios with Tim Iseler, then a guitarist with the TEAM AV-affiliated band . Both the tour and the recording engineer were arranged by the TEAM AV label, and the resulting "Like The Exorcist, But More Breakdancing" was released on Eyeball in August 2002. The band extensively toured behind the album with bands such as Cursive, Interpol, and The American Analog Set. In July 2003 the band released a split album with Volta Do Mar entitled "Konrad Friedrich Wilhelm Zimmer" in honor of the grandfather of the TEAM AV label owner who appears on the cover. Along with the song "Canyon Inn, Room 16" from the "Little Joe Gould" EP, Murder By Death also contributed "Knife Goes In, Guts Come Out" and the instrumental track "We Watch a Lot of Movies" alongside an alternative version of "A Masters in Reverse Psychology". "Konrad Friedrich Wilhelm Zimmer" was recorded with producer D. James Goodwin during sessions that would ultimately bear the release of "Who Will Survive, and What Will Be Left of Them?" in October. In December the band released a tribute 7" single for friend and musician Matt Davis of the band Ten Grand who had died earlier that year, before continuing to tour nationwide with Lucero, The Weakerthans, William Elliott Whitmore, and Rasputina.