Spotted by Cheltenham Town scout Derek Bragg playing for Carterton Town, Duff was invited by Mike Davis, youth team manager to join Cheltenham, then playing in the Southern League. He had a spell on loan at Cirencester Town before returning to Cheltenham when Steve Cotterill took over as manager. He became a regular for Cheltenham, and went on to play 242 games and score 15 times, including a 93rd-minute winner in a 3–2 victory at home to Yeovil Town in 1999 that secured Cheltenham Town promotion to the Football League.