Lord Lovat's involvement in politics continued throughout his life, in the House of Lords where he spoke on Scottish Affairs and served in the Inverness County Council for the next forty-two years where he pressed for modernising improvements. He also devoted much of his time to the family estates of 250,000 acres in the highlands and to Fraser Clan affairs. He bred a pedigree herd of shorthorn cattle and was an international judge of cattle travelling widely to Canada, America, Latin America and Australia in that regard. He lectured on agronomy and loved racing. He was chieftain of Lovat Shinty Club, the local shinty team which bears his family name. Lord Lovat experienced a great deal of sadness in his final years; Two of his sons predeceased him in accidents within days of each other. In 1994, a year before his death, the family's traditional residence, Beaufort Castle, was sold by his eldest son, Simon Fraser.