Feudal barony of Dunster Ryce, by whom he had no sons, only three daughters, Catherine, Dorothy and Mary, co-heiresses to 1/3 in total of his estate, the remaining 2/3rds going by entail to his younger brother Thomas Luttrell (died 1571). Mary survived him and remarried to James Godolphin of Cornwall. Thomas Luttrell (died 1571) of Dunster Castle (younger brother). In 1563 he was MP for the newly created Luttrell Pocket Borough of Minehead, two miles north-east of Dunster Castle. He sold the Devon and Somerset estates, excepting Dunster Castle, apparently to meet debts. These were however "amply replaced" by the large estate inherited from
Dunster Castle left in the castle grounds, the remaining servants had little to do. John's brother Henry inherited in 1857, but he too lived in London rather than at Dunster. George Luttrell inherited the castle in 1867 and began an extensive modernisation, backed by the considerable income from the Dunster estates – in a period of agricultural boom in England, the estates were producing £22,000 in revenue a year (£1.49 million at 2010 prices). It was fashionable during the mid-Victorian period to remodel existing castles to produce what was felt to be a more consistent Gothic or sometimes Picturesque appearance and George,