Williams was born in Bangor in North Wales and began his career as a Wolverhampton Wanderers apprentice. However, things did not work out and he moved back to Anglesey and joined League of Wales side Caernarfon Town, while working on a building site. Williams scored freely for Caernarfon and this attracted the attention of bigger clubs. On the eve of the 1997–98 season, Williams left Caernarfon for Barry Town, then the only professional side in the League of Wales, breaking their transfer fee in the process at a cost of £25,000. He immediately began scoring regularly, including one against Dynamo Kiev in the UEFA Champions League.