Gower, who never married, was well known in the homosexual community of the time. Oscar Wilde's story "The Portrait of Mr. W. H." has been interpreted as a comment on Gower's social circle, and Gower is generally identified as the model for Lord Henry Wotton in "The Picture of Dorian Gray". In 1879, hints of his homosexual liaisons published in the journal "Man of the World" led Gower to sue the paper, but later in the year the Prince of Wales sent him a letter accusing him of being "a member of an association for unnatural practices", to which Gower wrote an angry reply.