After Allen's death in 1764, William Warburton, Allen's relative, lived in the house for some time and it was passed down to other family members and then purchased, in 1809, by John Thomas, a Bristol Quaker. After William Beckford sold Fonthill Abbey, in 1822, he was looking about for a suitable new seat, Prior Park was his first choice: ""They wanted too much for it," he recalled later; "I should have liked it very much; it possesses such great capability of being made a very beautiful spot." Prior Park was offered for sale after Thomas's death in 1827 but the asking price of £25,000 was not obtained and the offer of sale withdrawn.