Patton was born in Hinds County, Mississippi, near the town of Edwards, and lived most of his life in Sunflower County, in the Mississippi Delta. Most sources say he was born in April 1891, but the years 1881, 1885 and 1887 have also been suggested. Patton's parentage and race also are uncertain. His parents were Bill and Annie Patton, but locally he was regarded as having been fathered by former slave Henderson Chatmon, several of whose children became popular Delta musicians, as solo performers and as members of groups such as the Mississippi Sheiks. Biographer John Fahey described Patton as having "light skin and Caucasian features."