The castle fell into decline in the 16th century; it was separated from the Duchy of Cornwall, and under Queen Mary the site was stripped for lead and other building materials for use at Windsor Castle. The antiquarian John Leland described the castle in 1540 as being "nowe sore yn ruine, and for the most part defaced", although the jail continued in use throughout the period, albeit still suffering from many escapee inmates. Held by various nobles from 1600 onwards, it returned to the crown under Charles I, who gave it to Queen Henrietta Maria, but by then the castle was only really valuable for the surrounding meadow land and fisheries.