"You Gotta Go There to Come Back" features a blues rock styled sound in the mold of early 1970s rock bands, and also displays influences of garage rock and soul. Kelly Jones described the album, with its "very 70s, Stevie Wonder, rock overdub feel", as the fulfillment of his desire to make an album like his favourite soul music: "I was really into soul music - it’s not something I’m ashamed about. I was brought up on Stevie Wonder and I love "Talking Book" and all the overdubs on it, and all that freestyling Marvin Gaye thing. I’d always wanted to make a record like that, and this was the first one I produced so that’s probably why I went 'Fuck it I’m just going to do it'." The "NME" felt that the album's "retro-garage" style made it "accidentally hip," comparing it to acclaimed contemporary garage rock revival "headbanger blues" bands like the White Stripes, a sentiment also shared by music critic Neil McCormick.