"Take This to Your Grave" was released in the United States on May 6, 2003, by Fueled by Ramen. The band held a release party at Chicago venue The Metro. Previously, one of the band's earliest recordings, "Evening Out with Your Girlfriend", had not been released until shortly before "Grave" in March 2003, when the band had gained considerable momentum. "Our record was something being rushed out to help generate some interest, but that interest was building before we could even get the record out," said Sean Muttaqi. The band actively tried to stop Uprising from releasing the recordings as the band's relationship with Muttaqi had soured. The band viewed it as a "giant piece of garbage" recorded before Hurley's involvement, and the band ceased to consider it their debut album.