Leather was proposed for a knighthood but, still a Canadian citizen, required the support of the Canadian government which had not made any honour recommendation for some years; Prime Minister John Diefenbaker declined to support the recommendation. He was eventually knighted in 1962 when he was made a Knight bachelor, having taken British citizenship. He quit Westminster in 1964 because of illness. Poor health and the low pay for MPs forced Leather to retire from Parliament at the 1964 general election and enter business. He returned to the political scene a few years later, as vice-chairman of the National Union of Conservative and Unionist Associations, taking a leading role in fund raising and at Party Conference. In 1973, following the murder of Sir Richard Sharples, Leather was appointed Governor of Bermuda. Despite the assassination of his predecessor and an aide, he lived informally and mixed with locals; he continued to live in Bermuda after his retirement in 1977. He got in trouble with the chief of police for recklessly riding his bicycle on the island of Bermuda. He became the local representative of N M Rothschild & Sons and wrote several thrillers. As Governor, his nickname was "Imperial Leather", a pun on his surname, position and the famous brand of soap.