The house was bought by Colonel Joseph Holden Strutt a British soldier and long-standing Member of Parliament. He renovated the house and neighbouring church. He extended the house before his death in 1845. His youngest daughter Charlotte Olivia inherited the house and lived there with her husband, the Reverend Robert Drummond, until her death in 1897. In 1912, it was inherited by Richard Strutt, the youngest son of the second Baron Rayleigh. Richard Strutt added an orangery and built the library based on designs by Charles Bateman. In addition, he joined the ruined chapel near the gate to the rest of the house. The house was then inherited by his son, John Strutt.