The earliest mention of the town itself is in the 12th-century "Chronicle of Bohemians" by Cosmas of Prague. He mentions the town of "Cladzco" as belonging to duke Slavník, father of Adalbert of Prague, in 981. Initially in Bohemia, together with the Kłodzko Land, it changed affiliation several times, passing between Poland and Bohemia in a series of conflicts which in turn devastated the town completely by the beginning of the 12th century. In 1114 Bohemian Duke Soběslav I captured and burnt the town to the ground, but he rebuilt it shortly afterwards. He also rebuilt and strengthened the castle located on a high rock overlooking the town. After the of 1137, Duke Bolesław III Wrymouth of Poland ceded all claims to the Kłodzko Land to the Bohemian Duchy (later Kingdom).