In February 1796 Foster returned once more to Wadsworth Lane, and remained there until early in 1797 he became minister of a general Baptist congregation at Chichester. About midsummer 1799 he removed to the house of an early friend, the Rev. Joseph Hughes, at Battersea, where he spent several months in preaching, and teaching twenty African boys whom Zachary Macaulay was training for mission work. In 1800 he took charge of a small congregation at Downend, near Bristol, and in February 1804 of one at Sheppard's Barton, Frome.