In May 1987, Smith was appointed manager at Sunderland, who had just been relegated into the Third Division for the first time in their history. York were unwilling to release him from his contract for less than £20,000 compensation, and Sunderland would only go as far as pay £10,000. Smith agreed to a £40,000-a-year contract that meant if he failed to win promotion with Sunderland in his first season he would pay the extra £10,000 compensation himself. He brought his York coaching staff with him: Viv Busby, Malcolm Crosby, and Roger Jones. He also signed former York player John MacPhail to shore up the defence, who captained the team and scored 16 goals (including 10 penalties) in his maiden season at Roker Park. He signed right-back John Kay, whilst remaining satisfied with existing left-back Reuben Agboola, centre-back Gary Bennett and goalkeeper Iain Hesford. In midfield he played a youthful combination of Gordon Armstrong, Paul Lemon, Gary Owers and Paul Atkinson, alongside the more experienced defensive midfielder Steve Doyle. He sold Mark Proctor to Sheffield Wednesday for £275,000. He signed young and pacey York striker Marco Gabbiadini for £80,000 and played him alongside the experienced Eric Gates; the pair scored 42 goals between them in 1987–88. A 7–0 win over Southend United started a run of 15 league matches unbeaten, though in the second half of the campaign a run of just 2 wins in 10 matches set them back. Smith bought Swansea City winger Colin Pascoe in March 1988, and oversaw a run of seven wins in the final eight matches to secure the Third Division championship and promotion with 93 points.