Schafer of "Stereogum" ranked the song eighth on his list of "The 10 Best Marilyn Manson Songs", saying that "While the remainder of "Portrait Of An American Family" comes across as undercooked, even for an early-'90s alt-metal album, 'Get Your Gunn' feels lean and vitriolic. Sadly, Manson seldom matched its aggression again." Schafer also praised Manson's "charming" vocal performance on the song. Jim Louvau of the "Phoenix New Times" named it one of his favorite Marilyn Manson songs alongside "Long Hard Road Out of Hell" (1997). Bloody Disgusting's Brad Miska opined that "In retrospect, its level of cheese is of legend, but in 1994 "Portrait" was some dark and weird shit (see 'Cake and Sodomy', 'My Monkey', 'Get Your Gun' , etc.)." "Gigwise" said that the song has "brutal" lyrics which "set the precedent for the rest of the band's career".