The castle was attacked during the Madog ap Llywelyn revolt of 1294, the Llywelyn Bren uprising in 1316 and during the overthrow of Edward II in 1326–27. In the late 15th century, however, it fell into decline and by the 16th century the lakes had drained away and the walls were robbed of their stone. The Marquesses of Bute acquired the property in 1776 and under the third and fourth Marquesses extensive restoration took place. In 1950 the castle and grounds were given to the state and the water defences were re-flooded. In the 21st century, the Welsh heritage agency Cadw manages the site as a tourist attraction.