It is likely that the settlement grew up on two sites: around the old manor house of Westbury, at the south end of the village, and around the old manor house of Croxton. It seems that a street to the east of the church formed the main village . By 1811 it had already reduced in size, and after enclosure in 1818, the rectory was demolished and an ornamental lake created on its site. By 1826, all the houses had gone and only Croxton Park mansion remained.