The genesis of "Run Baby Run" was in place by the end of 2003 at the band's home base of Smart Studios in Madison, Wisconsin. Duke Erikson had brought the idea for "Run Baby Run" into the sessions; drummer Butch Vig later described it as a song that was "really labored over". The song originally began with a heavy metal riff which ended up turning into a Cure-like verse part. Garbage tried speeding up the tempo, changing the groove, rewriting the vocal melodies and verse lyrics to support the chorus, which they were happy with. By the end of 2004, the band had essentially re-wrote the song, scrapping all the original verses. Bass guitar parts were performed by Justin Meldal-Johnson, while drums were performed by Matt Walker, as Vig wanted to control the sound of some of the songs from "behind the glass" during the album sessions, which were engineered by Billy Bush. The final recording was mixed by Vig at Smart Studios and mastered by Emily Lazar at The Lodge in New York City.