While at school, Gray had trials with Manchester United and signed schoolboy forms, but decided against signing a contract at Old Trafford and continued playing for Fatfield Juniors F.C. in the Hetton Youth League. After 18 months he signed, as an apprentice, with Sunderland. He broke into the Sunderland first team shortly after his 18th birthday, at the beginning of the 1992–93 season. He made his senior debut on 21 November 1992 in a 1–0 win at Derby County in Division One (the first season after the creation of the new Premier League).