Jones was born in Louisville, Kentucky. Jones began his career playing on a river boat named "Island Queen", which traveled between Kentucky and Ohio. He began in the 1920s playing on Mississippi riverboats and then in 1928 he joined with Horace Henderson. Later he worked with Jimmie Lunceford and had an early collaboration with Stuff Smith in 1932. From 1932 to 1936 he had a successful collaboration with Smith, but in the 1940s he worked in big bands like Benny Carter's and Fletcher Henderson's. He would spend most of a decade with Cab Calloway's band which later became a combo.