Jackson was born in Washington, D.C. on December 21, 1948. He grew up in Chattanooga, Tennessee. During childhood, he had a stuttering problem, which he conquered by developing an affinity for the use of the curse word, "motherfucker", in his vocabulary. Jackson attended Morehouse College in Atlanta, Georgia. After joining a local acting group to earn extra points in a class, Jackson found an interest in acting and switched his major. Before graduating in 1972, he co-founded the "Just Us Theatre". Jackson is a sports fan and supports basketball teams Toronto Raptors and the Harlem Globetrotters and soccer club Liverpool FC. In 1980, Jackson married LaTanya Richardson, with whom he has a daughter, Zoe.