Bunn was considered one of the best acoustic guitarists of the 1930s. He appeared on record for the first time in 1929 as a member of a trio with trumpeter Red Allen and pianist Fats Pichon, then as a guest with the Duke Ellington orchestra. Soon after, he recorded with the Six Jolly Jesters in the first of many washboard-and-kazoo sessions during his career with bands such as The Washboard Rhythm Kings and the Washboard Serenaders. During the following year, he participated in a rare session with Jelly Roll Morton and clarinetist Wilton Crawley. He also accompanied vocalists Victoria Spivey, Lizzie Miles, and Spencer Williams. For the rest of the decade, he recorded with the swing band Spirits of Rhythm led by vocalist Leo Watson.