He made 40 appearances in 1996–97 as Stoke finished in a mid-table position of 12th. In 1997–98 Stoke suffered relegation to the Second Division, however Whittle did manage to win the player of the year award despite playing in just 20 league matches. Under Brian Little in 1998–99 he played in 15 matches scoring his only goal for Stoke against Reading on 3 October 1998.
In 1996–97, Whittle made 40 appearances as Stoke finished mid-table at 12th place. In the subsequent season, 1997-98, Stoke were relegated to the Second Division but Whittle still won the player of the year award after playing in just 20 league matches. Under Brian Little's management in 1998–99, he played 15 matches scoring his only goal for Stoke against Reading on October 3, 1998.