Dunster Castle evidence of a fortification at Dunster was an Anglo-Saxon "burgh". This was built on the summit of the hill and was possibly intended to protect the region against sea-borne raiders; by the mid-11th century it was controlled by a local nobleman called Aelfric. In 1066 the Normans invaded south-east England, defeating the English forces at the battle of Hastings: in the aftermath of the victory, William the Conqueror entrusted the conquest of the south-west of England to his half-brother Robert of Mortain. Expecting stiff resistance, Robert marched west into Somerset, supported by forces under Walter of Douai, who entered from
Yarn Market, Dunster on the motte by the start of the 12th century, and the castle survived a siege during the early years of the Anarchy. At the end of the 14th century the de Mohuns sold the castle to the Luttrell family. Dunster had become a centre for woollen and clothing production by the 13th century, with the market dating back to at least 1222, and a particular kind of kersey or broadcloth became known as 'Dunsters'. The prosperity of Dunster was based on the wool trade, with profits helping to pay for the construction of the tower of the Priory Church