Sandra Schulman of the "Sun-Sentinel" said that the song is "Horrifyingly good, loopy music, with discordant riffs that crawl under your skin and stay there. The soundtrack to every nightmare you've ever had." "Dazed"s Daisy Jones described "Get Your Gunn", Nirvana's "Rape Me" (1993), The Prodigy's "Smack My Bitch Up" (1997) and Lady Gaga and R. Kelly's "Do What U Want" (2013) as "tracks that provoked, thrilled and shocked their listeners." Stephen Thomas Erlewine of AllMusic wrote that "Even though it wasn't one of the best tracks on "Portrait of an American Family", 'Get Your Gunn' was a solid track that was a fairly good choice for a single." Conversely, "The Chicago Maroon" Matt Zakosek called the track "weary Goth-rock junk" and criticized its inclusion on the greatest hits album "" (2004).