In opposition Boswell was a spokesman on the Treasury in the immediate aftermath of the 1997 election defeat and became a spokesman on Trade and Industry under William Hague, before speaking on Education and Employment in 1999 until after the 2001 general election. He became the spokesman for Work and Pensions under the leadership of Iain Duncan Smith, moving briefly to speak on Constitutional Affairs in 2003 under Michael Howard and back to Work and Pensions in 2004, where he remained following the 2005 general election.