After high school, Switzer went to the University of Arkansas. He played football there from 1956 to 1960. After graduation, Switzer join the US Army for a short time. After the Army, he returned to the University of Arkansas to work as an assistant coach. 