The site is adjacent to Malmesbury Abbey, which was founded in the 7th century and completed in its present form by the 12th century. The house was possibly begun in the 13th century as the dorter (domitory) and reredorter (latrine) of the abbey. In 1539, the abbey was sold by Henry VIII to a local clothier, William Stumpe, who also bought the site and lived in it himself. In 1542, Stumpe or his son James rebuilt the home in the Tudor style; the old section of the house remains mostly unchanged since then. The lower parts of the 13th-century building survive in the undercroft.