The castle was probably last used militarily during the time of the French occupation of Jersey (1461–1468, when the French held the east of the island, but loyal Jersey folk held the western parishes). In 1483, after several years of petitioning King Edward IV, the Seigneur of St Ouen - recommended to the King's Grace by his father-in-law the governor, was granted a ″License to Crenelate″ - that is allowed to fortify his manor house. Governor Harliston approved the dismantling of the ″fort of refuge″ at Grosnez so that the already-worked stone could be re-used in fortifying St. Ouen's Manor.
The castle probably stopped being used for military purposes during the French occupation of Jersey (1461–1468, when the French controlled the eastern part of the island, but local residents maintained control over the western parishes). In 1483, after years of lobbying King Edward IV, the Seigneur of St Ouen was granted permission to strengthen his manor house. Governor Harliston allowed the demolition of the ″safe haven″ at Grosnez to reuse its stone in strengthening St. Ouen's Manor.