As an adult, Walker enrolled at Oberlin College in 1878, where he majored in philosophy and the arts. At Oberlin, Walker proved himself to be an excellent student, especially in mechanics and rhetoric, but by his sophomore year, he was rarely attending classes. How Walker first came to play baseball is uncertain: according to Zang, the game was popular among Steubenville children, and while in Oberlin's preparatory program Walker became the prep team's catcher and leadoff hitter. Oberlin men played baseball as early as 1865—including a “jet black” first baseman whose presence meant Walker was not the college's first black baseball player—with organized clubs that engaged in intense matchups. Walker gained stardom and mentions in the school newspaper, "The Oberlin Review", for his ball handling and ability to hit long home runs.