Dunster Castle evidence of a fortification at Dunster was an Anglo-Saxon "burgh". This was built on the summit of the hill and was possibly intended to protect the region against sea-borne raiders; by the mid-11th century it was controlled by a local nobleman called Aelfric. In 1066 the Normans invaded south-east England, defeating the English forces at the battle of Hastings: in the aftermath of the victory, William the Conqueror entrusted the conquest of the south-west of England to his half-brother Robert of Mortain. Expecting stiff resistance, Robert marched west into Somerset, supported by forces under Walter of Douai, who entered from
Dunster Castle with a supply of venison as well as generating income. During the 15th century, England was divided by the prolonged period of civil war called the Wars of the Roses: the Luttrells were supporters of the House of Lancaster. In 1461, Sir James Luttrell died following the Lancastrian defeat at the Second Battle of St Albans, and his family were deprived of their estates by the Yorkist Edward IV. The castle was given to the Herberts, but the Luttrells regained it on the accession of the Lancastarian Henry VII in 1485, when Dunster was restored to James' son, Sir Hugh