Gordon often said that he wished he had been born a eunuch, which has been taken to suggest that he wanted to annihilate all of his sexual desires, indeed his sexuality altogether. Together with his sister Augusta, Gordon often prayed to be released from their "vile bodies" which their spirits were "imprisoned" in so that their souls might be joined with God. Faught argued that no-one at the time suspected Gordon of having sexual relations with the legions of teenage boys living with him at the Fort House, and the claim he was secretly having sex with the boys of the Fort House was first made by Lytton Strachey in his book "Eminent Victorians", which Faught commented may have said more about Strachey than it did about Gordon.