Construction of the fortress began in 1077 under Archbishop Gebhard von Helfenstein. The original design was a basic bailey with a wooden wall. In the Holy Roman Empire, the archbishops of Salzburg were already powerful political figures and they expanded the fortress to protect their interests. Helfenstein's conflict with Emperor Henry IV during the Investiture Controversy influenced the expansion of the fortress, with the Archbishop taking the side of Pope Gregory VII and the German anti-king Rudolf of Rheinfelden. The fortress was gradually expanded during the following centuries. The ring walls and towers were built in 1462 under Prince-Archbishop Burkhard II von Weißpriach.