Before becoming a royal residence, it was the centre of the Viceroys of Savoy that were appointed by Francis I of France after the French captured Savoy in 1536. The building later included the Bishops Palace called the "Palazzo San Giovanni".