Under Edward Heath, Prior was Minister of Agriculture, Fisheries and Food from 1970 to 1972, then Leader of the House of Commons until Heath lost office in the wake of the February 1974 election. His party returned to power under Margaret Thatcher in 1979, and Prior was Secretary of State for Employment from 1979 to 1981, disagreeing with some of her views on trade unions and her monetarist economic policies generally. This made him a leader of the so-called "wet" faction in the Conservative ranks. In 1981 he was moved to the less pivotal role of Secretary of State for Northern Ireland, from which he stood down in 1984; he never returned to government.