He was a Representative Peer for Scotland from 1952 to 1963, and in 1957, despite not being a member of the Conservative party, he was appointed Minister of State for Colonial Affairs in Harold Macmillan's government, a post he held until 1962. He was appointed a Privy Counsellor in 1957. Lord Perth was also First Crown Estate Commissioner from 1962 to 1977, Chairman of the Ditchley Foundation from 1963 to 1966 and Chairman of the Reviewing Committee on Export of Works of Art from 1972 to 1976.