Feudal barony of Dunster Ryce, by whom he had no sons, only three daughters, Catherine, Dorothy and Mary, co-heiresses to 1/3 in total of his estate, the remaining 2/3rds going by entail to his younger brother Thomas Luttrell (died 1571). Mary survived him and remarried to James Godolphin of Cornwall. Thomas Luttrell (died 1571) of Dunster Castle (younger brother). In 1563 he was MP for the newly created Luttrell Pocket Borough of Minehead, two miles north-east of Dunster Castle. He sold the Devon and Somerset estates, excepting Dunster Castle, apparently to meet debts. These were however "amply replaced" by the large estate inherited from
Dunster Castle ran up new debts, and the castle was handed over into the control of a receiver. Henry Fownes Luttrell, who married Margaret, Alexander's daughter, and took the Luttrell name, moved to Dunster in 1747. The couple redesigned and redecorated the castle in a Rococo style, including the extensive use of the recently invented and highly fashionable wallpaper. Henry Luttrell raised the ground height of the Lower Ward between 1764 and 1765 to extend the New Way all around to the front of his mansion, adding additional ornamental towers onto the inside of the Great Gatehouse in the process. A folly,