He had suffered from paralytic attacks, and had a severe stroke in 1811, from which time his faculties declined. He was invited to take up his abode in the house of the Literary Fund, 36 Gerrard Street, Soho, and there he remained till his death, regularly attending the society's meetings. At a special meeting of the general committee, held without Williams's knowledge on 25 July 1815, it was resolved to offer him £50 every six months, as evidence of the committee's 'attachment to the first principles of their society'. Only one instalment was paid before his death on 29 June 1816. A second instalment was handed to his niece and housekeeper, Mary Watkins. On 6 July he was buried in St Anne's Church, Soho, where there is a brief inscription to his memory.