From 1746 the estate was run by the Forfeited Estates Commissioners, appointed by Parliament to dispose of confiscated estates, and a small house was built on the site of the demolished castle to house the factor (estate manager). In 1774 the estate was returned to Lovat's son, Simon Fraser of Lovat (1726–1782), who had raised and commanded the 78th Fraser Highlanders for the British Army. Proposals for a new house on the site were put forward in 1777 but not executed. In 1815 the estate was inherited by Thomas Fraser of Strichen (1802–1875), who was reinstated to the Lordship of Lovat in 1854. In 1839 he commissioned William Burn to extend the house, and also improved the grounds and estate. His son Simon Fraser, 13th Lord Lovat (1828–1887), built the present Beaufort Castle, to designs by James Maitland Wardrop, incorporating part of the 18th-century house.