The estate was inherited in 1626 by Mary Talbot, sister of the 7th and 8th Earls from whom it passed to her husband, Sir George Savile, 2nd Baronet. He remodelled the house in 1685–95. Sir William Savile, 3rd Baronet, George's successor, made Rufford Abbey the seat of the Savile family after he burnt down the Saviles' original home to prevent its being occupied by a Parliamentarian garrison during the Civil War, but was killed in action in 1644. It was next inhabited by his son, the Marquess of Halifax, the Lord Privy Seal, who died in 1695. In 1679, he constructed a new north wing on the site of the abbey church, containing reception rooms and a long gallery. He also built the large stable block to the right of the house. The surviving service wing (currently used as offices) was also added by the Saviles in the 17th century.