Benson attended Clemson University from 1993 to 1996. His teammates included fellow future major-leaguers Billy Koch and Matthew LeCroy both of whom played with him in the 1996 Summer Olympics in Atlanta, Georgia, USA. (Koch reported teammates referred to Benson as "The Messiah".) In 1994, he played collegiate summer baseball for the Hyannis Mets of the Cape Cod Baseball League. Benson went undefeated during the regular season of his junior year (14–0 with a 1.40 ERA) with 178 strikeouts in 142 innings pitched.