Carn Brea Castle stands near the top of the hill. It is built on the site of a chapel built in 1379 probably dedicated to St Michael. It was built in the 18th century by the Basset family as a hunting lodge. It is considered to be a folly built on the huge uncut boulders that make up part of its foundations, giving the impression of the building melting into the land. An East India trading ship named after Carn Brea Castle, was wrecked off the Isle of Wight in 1829 and involved in excise tax fraud.