At a VCA meeting in April 1981, Loxton announced that he was severing all connections with organised cricket. He initially said that he was resigning for family reasons, as he would be moving to the Gold Coast in Queensland with his wife. However, he proceeded to give a blunt 15-minute speech, claiming that he had lost the art of communicating with the players and expressing his disenchantment with some aspects of the game, a reference to the declining player conduct. After relocating to the Gold Coast, he was unable to turn his back on cricket, and umpired matches at local level into his 70s. Although he became severely visually impaired, Loxton still attended matches and asked his companions to describe the proceedings for him; ever opinionated and blunt, he still offered advice to local cricketers.