In 1671 a window tax was introduced in the reign of Charles II, which inter alia recorded the population of Oakley as being 255 souls. In 1679 the Mordaunts sold Oakley to the Levinz family, who in their turn sold it in 1737 to the 4th Duke of Bedford. The Duke, a leading politician in George II's England, proved a most successful business man, and enlarged the Ducal properties considerably, buying Houghton House near Ampthill, as well as Oakley Manor House.