Settlement in the neighbourhood is of long standing. The Neolithic hillfort at King's Castle is east of the town. North west of the town is Clatworthy Camp, an Iron Age hill fort. Nearby is Elworthy Barrows, an unfinished Iron Age hill fort, rather than Bronze Age barrows. A rectangular enclosure south of Manor Farm is the remains of a Roman fort; in the 18th century its vestiges of fortifications and foundations were identified as being of Roman origin, and it was locally called "the Castle". In the 18th century a hoard of about 1600 Roman coins of third and fourth century dates was uncovered.