Banyard was frequently drunk until his wife asked him to attend a service in the local Wesleyan Methodist chapel. The preacher's message had a profound effect on Banyard, to the extent that he became teetotal and regularly attended the church. Before long he became a reputable preacher on the Wesleyan circuit. In 1837 he and William Bridges took a lease on an old workhouse at Rochford which became the first chapel of their new group, which Banyard and Bridges called the Peculiar People, a name taken from Deuteronomy 14:2 and 1 Peter 2:9.