The present house was built by the 3rd Earl of Bute to the designs of the neoclassical architect Robert Adam. Work commenced in 1767. The original plan had been for a grand and magnificent new house, but the plan was never fully executed and much of the work was a remodelling of the older house. Building work was interrupted by a fire in 1771 but by 1774 the house, though incomplete, was inhabited. Dr. Samuel Johnson, who visited the house in 1781, remarked: "This is one of the places I do not regret coming to see... in the house magnificence is not sacrificed to convenience, nor convenience to magnificence".