During the 12th century, the Abbot of Reading was Lord of the Manor of Hartley Dummer, which appears to have been around Grazeley. On the dissolution of Reading Abbey in 1541, Henry VIII granted the parish of Sulhamstead Abbots, and Grazeley with it, for purchase by Sir John Williams (later Lord Williams of Thame). After his death in 1559, his possessions were passed to his daughters. Through various sales and transfers, other major landowners declaring ownership of the area in their title deeds include the Norreyses of Rycote, the Earls of Abingdon, the Jameses of Denford and the Benyons of Englefield.