Descendants of Archibald Campbell, 1st Duke of Argyll sold the castle in 1801, to MacQuarrie, who then sold it to Carter-Campbell of Possil who kept it as a ruin within the grounds of his own estate to the north, Torosay Castle. He later sold his Torosay Estate which now included the ruins of Castle Duart to A. C. Guthrie in 1865. On 11 September 1911, the ruin was separated from the rest of the Torosay Estate and was bought by Sir Fitzroy Donald Maclean, the 26th Chief of the Clan MacLean and restored.