According to Nomina Villarum, by 1316 the Earl of Lancaster, and Robert de Saundeby, are certified to have been the lords of it. In 1355, nearly the whole of Babworth became the property of Sir Thomas de Grendon, who sold it in 1368 to Sir William Trusbutt. It was later inherited by Trussbutt's son, Sir Robert, who sold the manor "with its appurtenances" to Sir Richard de Willoughby, of Wollaton. Later it became the property of the Earl of Shrewsbury and Lord Cavendish, and in the 18th century it was purchased by Sir Gervas Elwes, and lastly by John Simpson.