Pitman was active in the local New Church congregation in Bath while living on Royal Crescent. He was one of the founding members, when this congregation was formed in 1841. He served as president of this society from 1887 to his death in 1897. His contribution to this church was honoured by the congregation with a stained glass window depicting the golden cherub in the temple of wisdom described in Swedenborg's "True Christian Religion" No. 508. The window was dedicated on 5 September 1909.