It is possible that the castle was never more than a wooden fortification, surrounded by a moat.
Given the information provided, it is plausible that the castle was not a grand and imposing structure but rather a simple wooden fortification with a moat for defense.