In summer 1806 Foster resigned the charge of the Sheppard's Barton congregation, troubled with a thyroid, and concentrated on writing. In May 1808 he married and went to live at Bourton in Gloucestershire. About a year after that his throat so far recovered as to allow him to resume occasional preaching, and towards the end of 1817 he again took charge of the congregation at Downend. In 1821 he gave it up and went to live at Stapleton, Gloucestershire. In 1822 he began to lecture fortnightly in Broadmead Chapel, Bristol; at the end of two years poor health forced him to make the lectures monthly, and in 1825, when Robert Halls began his ministry in Bristol, he dropped them.