In the second half of the 14th century the Weitmile family was significantly involved in the development of the town. Members of this family used to hold positions of reeve and parsons. Petr z Weitmile was assigned as a witness on the granting of the urban discretions document by Hynek Berka z Dubé in 1381. Development of the town and Bohemia was paused by a plague epidemic in 1389. At the end of the 14th century the castle was controlled by other members of lords of Lipá family, among whom a powerful and significant individual named Hynek Hlaváč was often mentioned, until the beginning of the Hussite wars, when in May 1426 it was conquered by Hussites led by Jan Roháč z Dubé, and became a stronghold until 1436. Between 1502 and 1553 a large part of the town and its surroundings belonged to the Vartenberks. Later, the lords of Dubá / Lipá regained the castle and kept it for over 100 years. The first evidence of Jewish settlement in the town dates back to 1562. Albrecht of Valdštejn reunited the town in 1622 to 1623. Large parts of the town were destroyed in 1787 and 1820.