Loch never entirely recovered his health after this experience in a Chinese dungeon. Returning home he was invested as a Companion of the Order of the Bath, and for a while was private secretary to Sir George Grey, 2nd Baronet, then at the Home Office. In 1863 he was appointed Lieutenant Governor of the Isle of Man. During his governorship the House of Keys was transformed into an elective assembly, the first line of railway was opened, and the influx of tourists began to bring fresh prosperity to the island. In 1882 Loch, who had become Knight Commander of the Bath in 1880, accepted a commissionership of woods and forests, and two years later was made governor of the colony of Victoria in Australia. In June 1889 he succeeded Sir Hercules Robinson as Governor of Cape Colony and High Commissioner for Southern Africa.