After high school, Walker went to Spelman College in Atlanta. She later moved to Sarah Lawrence College near New York City, graduating in 1965. Walker became interested in the U.S. civil rights movement. This was partly due to the influence of activist Howard Zinn, who was one of her professors at Spelman College. Walker returned to the South where she became involved with voter registration drives, campaigns for welfare rights and children's programs in Mississippi.