Skelton was due to defend his British title against his main rival Danny Williams in July 2005, but Williams pulled out at the last minute citing a case of the flu, a decision that prompted scorn from Williams' promoter Frank Warren. Skelton stopped late sub Mark Krence in 7 rounds, and scored a 1st-round knockout of John McDermott to finish the year, whereupon he vacated his British title, having defended it three times. In February 2006, Skelton finally fought Williams, and lost on split decision handing him his first pro defeat. In July 2006, he regained his Commonwealth title from Williams.