High Bradfield, which was formerly known as "kirkton", is not mentioned in the Domesday Book, but it may have been one of the 16 hamlets recorded in the manor of Hallam. On the northwest edge of the village, close to the church, is Bailey Hill, an approximately high man-made conical mound that is a Scheduled Ancient Monument. The date and purpose of its construction are unknown, though it has been interpreted as a Saxon fort, a Norman Motte-and-bailey castle, or a place of public village assembly. The late David Hey said there was no doubt that Bailey Hill is a motte-and-bailey castle calling it "One of the best preserved and most dramatic motte-and-baileys in Yorkshire." Excavations from 1720 revealed squared stones that had been produced by using tools. The eastern and southern flanks of the bailey are enclosed by a 95 metre long curving earthwork while to the west it is protected by steep slopes. About to the southeast of the village is Castle Hill, a site marked on old maps as a “supposed Saxon encampment” The site occupies a rocky ridge at the head of a high escarpment partially enclosed by ring work. The date or purpose of construction of this site are also unknown although it has been speculated that it was used a lookout post.