In 1842, Irving accepted a nomination as Ambassador to the Court of Isabella II of Spain. He left Sunnyside in the care of his brother Ebenezer, who lived there with his four grown daughters, who supervised the running of the household. Irving wrote, "The only drawback upon all this is the hard trial of tearing myself away from dear little Sunnyside." He returned to New York on September 19, 1846. Shortly after his return, in 1847, he added to the cottage the "Spanish Tower", influenced by Spanish monastic architecture and the Alhambra in Granada. It added four bedrooms to the house.