Torbush was part of some mild success at Ole Miss, where he was defensive coordinator from 1983 to 1986. The 1986 season saw Ole Miss compile an 8–3–1 record including a season-ending 20–17 win at the Independence Bowl over Texas Tech. The 1986 Rebel defense allowed opponents an average of less than 13 points per game and statistically ranked as the best defense in the Southeastern Conference. It was after that 1986 season that Torbush was hired away to become head coach at Louisiana Tech.