In the reign of Henry VIII, the estate was granted to Thomas Howard, 2nd Duke of Norfolk, who destroyed the original structure and erected a magnificent new building with two fronts. The house and estate passed to Thomas Howard, 3rd Duke of Norfolk. The estate was confiscated by the Crown when he was arrested on suspicion of treason. The house served as a residence for both of Henry VIII's daughters: Princess Mary and Princess Elizabeth at different times during the reign of Edward VI. When Mary became Queen in 1553, she granted the estate to the 3rd Duke's grandson, Thomas Howard, 4th Duke of Norfolk.