Feudal barony of Dunster Fownes (c. 1722 – 1780), who under his father-in-law's will inherited the Luttrell estates including Dunster Castle, with the proviso that he should adopt the surname and arms of Luttrell. Henry Fownes (c. 1722 – 1780), who married the heiress Margaret Luttrell (1726–1766) and under his father-in-law's will inherited the Luttrell estates including Dunster Castle, with the proviso that he should adopt the surname and arms of Luttrell. He was High Sheriff of Somerset from 1754 to 1755, and a Member of Parliament for Minehead from 1768 to 1774. The couple modernised the Castle, in the Georgian style, which included
Dunster Castle be slighted to prevent their further use. In the 1860s and 1870s, the architect Anthony Salvin was employed to remodel the castle to fit Victorian tastes; this work extensively changed the appearance of Dunster to make it appear more Gothic and Picturesque. Following the death of Alexander Luttrell in 1944, the family was unable to afford the death duties on his estate. The castle and surrounding lands were sold off to a property firm, the family continuing to live in the castle as tenants. The Luttrells bought back the castle in 1954, but in 1976 Colonel Walter Luttrell gave Dunster