In addition to Tutbury Castle, he built two more, probably typical Norman timber motte and bailey construction. Remembering that large areas of the county were laid to waste during the so-called Harrying of the North, and are recorded as such in the Domesday Survey, Pilsbury Castle, on the west bank of the River Dove, was probably built to protect his holdings in the wapentake of Hamston. Meanwhile Duffield Castle commanded an important crossing over the River Derwent and oversaw the parts of the wapentakes of Litchurch and Morleyston, to the west of the river, and that part of his lands that would become the Frith. Much of the estate was granted to Knights who served under him, among them being the Curzons of Kedleston Hall.