James was born near Bentonia, Mississippi. He worked on road construction in the early 1920s, and wrote what his earliest song, "Illinois Blues", about his experience as a worker. He began playing guitar in open D-minor tuning with three fingers.