Duff was one of the most powerful and influential men in Aberdeenshire, Banffshire and Morayshire due to his massive estate. Due to his political commitments in Westminster, the running of the Estate was largely left to his factors - in particular William Rose of Ballivant. For his service to Duff, Rose was awarded many political favours. First, Duff gave Rose a vote in three different constituencies (Aberdeenshire, Banffshire, Morayshire), he appointed him the first Provost of Macduff, and latterly the Sheriff Clerk of Banffshire.