Some historians have accused him of double-dealing in swearing allegiance to William III while plotting the restoration of James; others argue that his true loyalty was to the institution of the monarchy, and that he supported whichever monarch seemed best fitted to rule at any given time. William III clearly did not regard him as a dangerous character, as shown by the fact that he was left in peace once he fled from England; he was fortunate in having a great many friends and very few enemies. It seems that from about 1710 he was free to return to England, but he was by then happily settled in Brussels, where he had made a second marriage for love to Charlotte, comtesse d'Esneux, and, since he was able to draw at least part of the revenue from his English estates, he had no pressing need and no apparent desire to return home.