The manor of Milton ("Mildeltune") is listed in the Domesday Book of 1086 and literally means "Middle farm." It was part of the lands belonging to Hugh de Port, and the estate was held from him by William Chernet. The Chernet family maintained possession of Milton into the 13th century, although lesser families were managing the estate on their behalf. The most important of these were the Chaucombe (or Chalcombe) family, who were probably the first people to build a church in Milton in the mid 13th century. In 1303, Thomas de Chaucombe was given permission to hold a weekly market on Tuesdays at Milton, as well as an annual fair on the feast day of Mary Magdalene, but this attempt to create a market town seems to have failed.