Hvedholm Castle was later returned to the Brahe family, who were considered for generations the rightful rulers of the castle and surrounding villages.
Afterwards, Hvedholm Castle was returned to the Brahes, a family considered the legitimate rulers of the castle and its adjacent communities for centuries.