Born in Oldham, Greater Manchester, Bywater attended the Blue Coat School, Oldham as a boy starting his career in August 1997 with Rochdale. His only first team appearance was against Carlisle United in an Football League Trophy game which Rochdale lost 6–1. He was brought to West Ham as a 16-year-old by Harry Redknapp in February 1998 for £250,000, rising to £1.75 million depending on performances and achievements. He graduated through the youth set-up and featured in West Ham's 1999 winning team along with Joe Cole and Michael Carrick.