By 1801, Lambert's weight had increased to about , and, as his bulk meant neither he nor his horse were able to keep up with the hunt, he was forced to give up hunting. He continued to maintain an interest in field sports, keeping a pack of 30 terriers. By this time, although he retained his solid reputation as a gaoler, serious concerns were being raised about his fitness for the post. Traditional gaols were falling out of favour and being replaced with forced labour institutions, and in 1805, the old Bridewell gaol was closed. Lambert was left without a job, but was granted an annuity of £50 (about £ as of ) a year by the Leicester magistrates, in recognition of his excellent service as gaol keeper.