Brown moved into an on-campus boarding house at 61 East Eleventh Avenue in the primarily black neighborhood of the University District in Columbus. He majored in architectural engineering. Brown attempted several times to apply to the school's aviation program, but was denied because of his race. Brown joined the track and field team as well as the wrestling team, but soon dropped both for financial reasons. He took a job as a janitor at a local Lazarus department store and was hired by the Pennsylvania Railroad to load boxcars from 15:30 to midnight each day. In spite of this, he maintained top grades in his classes.