It was transformed in a castle during the Middle Ages and the building was abandoned very soon to become the nest of bandits who exploited the straddle position between the two branches of the lake. As confirmation of this – in this regard Balbiani writes – we are aware that in 1375 Gio. Galeazzo Visconti, Filippo’s father, ordered the construction of a nice fortress with square stones which rose on the peak, and that it became an infamous shelter for bandits”. Remaining from the original fortress was a great square tower which was successively restructured by Ercole Sfondrati. In the XV Century, the summer residence of known Milanese Humanist was built on the castle’s ruins: Daniele Birago, bishop of Mytilene, commendatory abbot of the Abbey of Piona, on the other side of the lake. In 1489, the building and the vast lands surrounding it were ceded to the young Marquis Stanga, treasurer and ambassador of Ludovico il Moro.