Feudal barony of Dunster Ryce, by whom he had no sons, only three daughters, Catherine, Dorothy and Mary, co-heiresses to 1/3 in total of his estate, the remaining 2/3rds going by entail to his younger brother Thomas Luttrell (died 1571). Mary survived him and remarried to James Godolphin of Cornwall. Thomas Luttrell (died 1571) of Dunster Castle (younger brother). In 1563 he was MP for the newly created Luttrell Pocket Borough of Minehead, two miles north-east of Dunster Castle. He sold the Devon and Somerset estates, excepting Dunster Castle, apparently to meet debts. These were however "amply replaced" by the large estate inherited from
Dunster mentioned in the Domesday Book of 1086. From that time it was the "caput" of the Feudal barony of Dunster. The Castle was remodelled on several occasions by the Luttrell family who were lords of the manor from the 14th to 20th centuries. The benedictine Dunster Priory was established in about 1100. The Priory Church of St George, dovecote and tithe barn are all relics from the Priory. The village became a centre for wool and cloth production and trade, of which the Yarn Market, built by George Luttrell (d.1629), is a relic. There existed formerly a harbour, known as