It was originally the site of a medieval hunting lodge. Philip II built it as a modest summer palace. The palace was designed by Juan Bautista de Toledo and Juan de Herrera who also designed El Escorial. Later kings kept adding to it. It was completed during the reign of Ferdinand VI in the mid-18th century. Charles III added two wings to it. By that time the palace had over 300 rooms.