It is not known who built Frauenstein Castle, but, based on dendrochronological evidence, its construction has been dated to around 1184. Built on a quartzite outcropping of the "Spitzen Stein" the castle was not intended as a defensive fortress, but rather as a watch-tower. It also served the tower guards as a dwelling. The construction of the castle with its high walls and battlements gave protection to farmers in the area and led to the emergence of the village of the same name. Later, the castle was extended by a fore-castle with pond, as could be confirmed by excavations in the year 1943.