Feudal barony of Dunster Ryce, by whom he had no sons, only three daughters, Catherine, Dorothy and Mary, co-heiresses to 1/3 in total of his estate, the remaining 2/3rds going by entail to his younger brother Thomas Luttrell (died 1571). Mary survived him and remarried to James Godolphin of Cornwall. Thomas Luttrell (died 1571) of Dunster Castle (younger brother). In 1563 he was MP for the newly created Luttrell Pocket Borough of Minehead, two miles north-east of Dunster Castle. He sold the Devon and Somerset estates, excepting Dunster Castle, apparently to meet debts. These were however "amply replaced" by the large estate inherited from
Feudal barony of Dunster War he made Dunster Castle available for use as a convalescent home by the Royal Navy. On the death of his father in 1944 the payment of a large sum in Death Duties left the estate indebted and uneconomic to operate, and Geoffrey, whilst reserving a tenancy for himself at the Castle, sold the estates to a property development company which sold them on to the Commissioners of Crown Lands. He bought back the Castle and grounds in 1954 and opened them to the paying public. He died in 1957 and his widow remained at the Castle until her death