In 1590-1591 during the French Wars of Religion the castle came under siege and parts of towers and walls were destroyed. Subsequently, Louis XIV failed to maintain the building and it fell into a state of disrepair. At the beginning of the 18th century the arm of the Rhône flowing between the castle and the right bank silted up so that instead of sitting on an island the castle now stood on the bank of the river near the town. After the revolution the ruined castle was sold off in lots and used as a source of stone for other buildings.