On his death in 1797, possession passed to the family of his sister, Lady Mary, Countess of Cholmondeley, who had died aged just 26 more than 65 years earlier in 1731. She had married George Cholmondeley, 3rd Earl of Cholmondeley and Houghton Hall has been modified and maintained by her Cholmondeley family descendants to the present day. Colonel Robert Walpole borrowed a book about the Archbishop of Bremen from the Sidney Sussex College library in 1667 or 1668. The overdue library book was discovered at Houghton in the mid-1950s, and returned 288 years late.