His mother, Lottie, was a school teacher and his father, Kennell, was a building contractor. He attended Robert Russa Moton High School and graduated in 1958. He later received his bachelor's degree in Biology and Chemistry in 1962 from the Hampton Institute. He graduated second in his class and was awarded special distinction for dissertation. He later received his Ph.D. from UCLA in 1970. After graduating, Jackson became an expert in East Africa and African American cultural history. He was a fan of collecting books, as well as modern art.