It was the site of Bruton Abbey, a medieval Augustinian priory from which a wall remains in the Plox close to Bow Bridge. The priory was sold after the dissolution of the monasteries to the courtier Sir Maurice Berkeley (died 1581), whose Bruton branch of the Berkeley family converted it into a mansion then demolished in the late 18th century. The Dovecote which overlooks Bruton was built in the sixteenth century. It was at one time used as a house, possibly as a watchtower and as a dovecote. It is a Grade II* listed building and ancient monument. It is managed by the National Trust. The building was once within the deerpark of the Abbey. It was adapted by the monks from a gabled Tudor tower. The conversion to be a dovecote took place around 1780. It has over 200 pigeon holes.