Good times came to an end in the first half of the 17th century. In 1630 Szydłów was burned by rebellious soldiers of fortune, who were not paid their salaries. Unable to enter the fortified town, the mercenaries set the suburbs on fire, and the blaze reached Szydłów. Swedish invasion of Poland (1655–1660), and the invasion of troops of George II Rakoczi (see Treaty of Radnot) brought almost complete destruction, after which town's population shrank from 1,300 to 350, and the number of houses was reduced to 54 (as for 1663). After these conflicts, Szydłów never recovered to its previous prosperity. In 1789, Szydłów had a stone town hall, 5 mills on the Ciekąca river and 196 houses, many of them abandoned. The castle was ruined, and by 1827, the number of houses grew to 202, and the population reached 1,550.