Hadleigh is known for its castle, and the country park to the south of the town centre. This gives its name to the local government district of Castle Point, with its extensive views overlooking the Thames Estuary. Most of the facing stones were stripped from the castle in the 16th century – the only bits still visible today being high inside the surviving towers and a small section of the 'gate house' – so most of what remains today is the rubble infill that was packed between the outer facing stones. Despite this the skeletal remains are pleasing to the eye and have been considered a romantic ruin for a few hundred years. John Constable painted Hadleigh Castle in 1829. The painting now belongs to the Yale Center for British Art, and is on permanent display in its museum on the Yale campus.