Until the publication of the "Memoirs" Hervey was chiefly known as the object of savage satire on the part of Alexander Pope, in whose works he figured as Lord Fanny, Sporus, Adonis and Narcissus. The quarrel is generally put down to Pope's jealousy of Hervey's friendship with Lady Mary Wortley Montagu. In the first of the "Imitations of Horace", addressed to William Fortescue, Lord Fanny and Sappho were generally identified with Hervey and Lady Mary, although Pope denied the personal intention. Hervey had already been attacked in the "Dunciad" and the "Peribathous", and he now retaliated. There is no doubt that he had a share in the "Verses to the Imitator of Horace" (1732) and it is possible that he was the sole author. In the "Letter from a nobleman at Hampton Court to a Doctor of Divinity" (1733), he scoffed at Pope's deformity and humble birth.