A sixth-place finish in 2003–04 resulted in Grays being placed in the new Conference South for the 2004–05 season. They went on to win the new division in its first season, earning promotion to the Conference National. The season also saw them reach the final of the FA Trophy, in which they beat Hucknall Town 6–5 on penalties after a 1–1 draw at Villa Park. In their first season in the Conference National the club finished third, qualifying for the promotion play-offs. However, they lost 5–4 on aggregate to Halifax Town. They also reached the second round of the FA Cup, beating York City 3–0 at Bootham Crescent before losing 3–0 at Mansfield Town, as well as retaining the FA Trophy with a 2–0 win over Woking in the final. The club's league form subsequently faded and after narrowly avoiding relegation in 2006–07 and 2008–09, they finished bottom of the Conference National in 2009–10 and resigned from the Football Conference.