First mentioned in the mid C12. c.1200 documents mention fishponds already in use. The acquisition of timber for building work is recorded in 1213, and a garden in 1292. In 1304 Thomas de Latimer was granted a licence to strengthen his manor house at Braybrooke and documentary sources indicate that the moated house was constructed at this time. In 1329-30 there is mention of a waterfilled moat. Described as a castle in 1361. The manor passed to the Griffin family in the early C15. The castle buildings were used as a farmhouse from 1549-50 until demolition prior to 1633. A mid-C17 farmhouse stood on the site until 1960.