The castle was captured and destroyed by Llywelyn ab Iorwerth of Gwynedd in 1223. It was returned under the peace treaty, and was rebuilt in stone, replacing the tower keep of a motte and bailey with inner bailey with buildings along a curtain wall and five towers on a raised platform surrounded by a moat, beyond which there is an outer gatehouse or barbican. For the next forty years, the castle remained in English possession, but was ceded to Llywelyn ap Gruffudd in 1267, remaining in Welsh possession until 1276.