During the Cromwellian invasion of 1650, it is said the castle occupants suffered a great deal and many were executed. The castle and surrounding land was acquired soon after by the Rochfort family, who became the Earls of Belvedere. In 1796 the 2nd earl leased the castle to the army as a barracks. In 1850 ownership passed to Charles Brinsley Marlay, grandson of the 1st earl's only daughter, after whom it passed to Lieut.-Colonel Charles Howard-Bury, the explorer. He sold off the houses on the estate.