Vernon Dahmer was born on March 10, 1908, in the Kelly Settlement, Forrest County, Mississippi to Ellen Louvenia (Kelly) and George Washington Dahmer. George Dahmer was described as an honest, hardworking farmer of integrity. His mother Ellen Kelly was biracial because of her mother, Henrietta. Henrietta was a biracial child born out of wedlock by a white slave owner, O.B Kelly, and one of his slaves. She was given to a black family, called the McCombs. His cousin, Iola Williams, became the first African-American member of the San Jose, California, City Council in 1979.