In the 2001–02 season, Sunderland struggled, and with their dip in form came a dip in form for Gray, and for the next two seasons, he was constantly rumoured to be moving elsewhere including to arch-rivals Newcastle United. Relegation came for Sunderland at the end of the 2002–03 season, and two games into the 2003–04 season Gray finally left, joining Celtic on 1 September 2003, on a four-month loan. However, having failed to make an impact, and generally finding himself out of the team, he moved to Blackburn Rovers on a free transfer on 1 January 2004.