In 1967, Colville was made an Officer of the Order of Canada, elevated to Companion in 1982, the order's highest level. He lived in St. Catharines, Ontario, for three years before moving to Nova Scotia. In 1973, he moved his family to his wife's hometown of Wolfville, where they lived and worked in the house that her father built and in which she was born. The Colvilles had three sons, a daughter, and eight grandchildren. In contrast to many of his contemporaries, Colville aligned himself with the Progressive Conservative Party of Canada and was a card-carrying party member for many years. In 1981 he was appointed university chancellor of Acadia University serving in that role until 1991.