In 1752, the buildings were bought by the Damer family: in 1771, to make way for a new house and landscaped estate, the 1st Baron Milton (later 1st Earl of Dorchester) demolished the remaining abbey buildings, keeping only part of the church as a private chapel, and the adjacent market town of Milton (creating Milton Abbas to rehouse the former inhabitants) in 1780. The new house was designed by William Chambers and the gardens by Capability Brown. Several members of the Damer family were buried in the family vault in the Abbey Church.