In April 1947 he entered the cabinet as Secretary of State for India and Burma. Prime Minister Clement Attlee, however, made all the government's major decisions regarding India. After India gained independence in August 1947, his cabinet title became Secretary of State for Burma, working from the Burma Office, but in January 1948 this too was abolished, when Burma also gained independence, and Listowel then left the cabinet. He continued to serve under Attlee as Minister of State for Colonial Affairs from 1948 to 1950 and as Joint Parliamentary Secretary to the Ministry of Agriculture and Fisheries from 1950 to 1951. In 1957 he was appointed Governor-General of Ghana, a post he held until 1960, when Ghana became a Republic. He was later Chairman of Committees in the House of Lords between 1965 and 1976. He remained an active member of the House of Lords, speaking for the last time in July 1995, aged 88.