Drew won an athletics scholarship to Amherst College in Massachusetts, from which he graduated in 1926. An outstanding athlete at Amherst, Drew also joined Omega Psi Phi fraternity as an off-campus member; Amherst fraternities did not admit blacks at that time. After college, Drew spent two years (1926–1928) as a professor of chemistry and biology, the first athletic director, and football coach at the historically black private Morgan College in Baltimore, Maryland, to earn the money to pay for medical school.