Rushing was inspired to pursue music and sing blues by his uncle Wesley Manning and George "Fathead" Thomas of McKinney's Cotton Pickers. He toured the Midwest and California as an itinerant blues singer in the early 1920s before moving to Los Angeles, where he played piano and sang with Jelly Roll Morton. He also sang with Billy King before moving on to Walter Page's Blue Devils in 1927. He and other members of the Blue Devils defected to the Bennie Moten band in 1929.