Llywelyn the Great captured it in 1215 and at the parliament held at Aberdyfi in 1216 made it over to the sons of Gruffydd ap Rhys of Deheubarth, but in 1223 William Marshall the Younger recaptured it. In 1231 the castle was again captured for Llywelyn by Rhys Gryg and his allies. Llywelyn held it until his death in 1240. On Llywelyn's death it fell back into Norman hands, and in 1244 Earl Gilbert of Pembroke rebuilt it with town walls for added protection. It is the remains of this building that still stands overlooking the river.