He entered his sophomore season as the backup quarterback to Justin Zwick, but took over as the starter when Zwick was injured halfway through the 2004 season against the Iowa Hawkeyes. Smith won four of the five games he started in 2004, including a victory over the archrival Michigan Wolverines. Smith was suspended for breaking an undisclosed team rule before the Alamo Bowl, with the NCAA extending the suspension to include the first game of the 2005 season after it was revealed that Smith had accepted $500 from a booster.