Foster started his career as a trainee with Sheffield Wednesday, signing a professional contract with the club at the age of 18. He joined Scarborough on work experience in September 2004, and was also loaned out to Conference North side Alfreton Town in February 2005 until the end of the 2004–05 season. On returning to his parent club, Foster was released, and joined Lincoln City in June 2005. He was loaned out to York City the following season on a two-month loan. In January 2007, Foster was released by Lincoln, and he joined Stalybridge Celtic on a short-term contract. He left Stalybridge a month later, in order to join Oxford United on an initial six-month deal. Foster went on to play over 100 times for Oxford in just under three years at the club, winning two end of season awards in the process and was linked with a move Stoke City in national newspapers but after the club quickly secured Foster on a new 2-year extension with reportedly doubling his wages the rumours went quiet.