From May 2004 to June 2005 Smith played six Test matches, failing to reach a half-century in that time. During the 2004 tour of England he played three ODIs in the NatWest Series, where he made a highest score of 44, it was his last appearance for the ODI team until January 2007. Smith made a slight return to form in November 2005 against Australia in Brisbane. In a heavy defeat he was the only West Indies player to score a half century, making 88 in the first innings. Despite this Smith did not perform well in the rest of the series and again lost his place in the side.