Coleman was born to Laura Beatrice (née Mason) Coleman and William Thaddeus Coleman Sr. in Germantown, Philadelphia, Pennsylvania. Coleman's mother came from six generations of Episcopal ministers, including an operator of the Underground Railroad. W.E.B. DuBois and Langston Hughes would visit the family's home for dinner. One of seven black students at Germantown High School, Coleman was suspended for cursing at a teacher after she praised his honors presentation by saying, "Someday, William, you will make a wonderful chauffeur." When Coleman attempted to join the school's swim team he was again suspended, and the team disbanded after he returned so as to avoid admitting him, only to reform after he graduated. Coleman's swim team coach wrote him a strong letter of recommendation and he was accepted into the University of Pennsylvania, where he was a double major in political science and economics.