There is apparently some confusion as to whether Rilston's manor house stood on the same site occupied by the present Woolley Hall. J.W. Walker, in his work on the Manor and Church of Woolley, seems to have assumed that Sir Richard Woodrove made Rilston's house his principal residence on acquiring it in 1490. However, Geoffrey Markham argues that Woodrove may have remained in his ancestral home in Woolley, itself a house of some importance, and that it may be this house, previously owned by Sir William de Notton and John Woodrove, that stood on or near the site of the present Hall.