Chavis joined Tennessee's football squad as a walk-on defensive lineman. He played sparingly in 1976, picking up two tackles. He delivered an explosive performance in the 1977 Orange-and-White Game, and briefly pushed future All-SEC lineman Jim Noonan for the starting spot at middle guard. Playing as a backup to Noonan, Chavis registered 40 tackles (26 solo) and two sacks, and was awarded a scholarship at the end of the year. Prior to the 1978 season (his senior year), Chavis switched to defensive tackle, and was vaulted into the starting slot after Bill Christian suffered a season-ending knee injury. He finished the year with 32 tackles (19 solo).