Wellesley lived together for many years with Hyacinthe-Gabrielle Roland, an actress at the Palais Royal. She had three sons and two daughters by Wellesley before he married her on 29 November 1794. He moved her to London, where Hyacinthe was generally miserable, as she never learned English and she was scorned by high society: Lady Caroline Lamb was warned by her mother-in-law, Elizabeth Milbanke, a noted judge of what was socially acceptable, that no respectable woman could afford to be seen in Hyacinthe's company.