Dunster Castle evidence of a fortification at Dunster was an Anglo-Saxon "burgh". This was built on the summit of the hill and was possibly intended to protect the region against sea-borne raiders; by the mid-11th century it was controlled by a local nobleman called Aelfric. In 1066 the Normans invaded south-east England, defeating the English forces at the battle of Hastings: in the aftermath of the victory, William the Conqueror entrusted the conquest of the south-west of England to his half-brother Robert of Mortain. Expecting stiff resistance, Robert marched west into Somerset, supported by forces under Walter of Douai, who entered from
Feudal barony of Dunster who was a "powerful personage" due to her large Wyndham dowry, purchased Dunster Priory following the Dissolution of the Monasteries, which thenceforth descended with the castle. By his wife Margaret Wyndham he had children including three sons: Sir John Luttrell (died 1551), of Dunster Castle, eldest son and heir. He was one of the principal commanders in the Scottish war under the earl of Hertford. In 1545 he was knighter after the capture of Leith. He was taken prisoner by the Scots in 1550 at Broughty Craig and was ransomed for £400. He married Mary Ryce, daughter of Sir Griffith