Aubrey II and Aubrey III are candidates for initiating the construction of a major stone tower at Hedingham, possibly to reflect the enhanced status of the family. In 1133 Aubrey II, son and heir of the first Aubrey, was created master chamberlain of England by Henry I. In 1141, his son and heir Aubrey was granted an earldom by Empress Matilda. By that time he had been Count of Guines for several years by right of his wife's inheritance of that continental territory.