On 25 June 1874, he met and married the American Mary Crawford in Italy. Mary was the daughter of prominent sculptor Thomas Crawford, and the sister of fellow author Francis Marion Crawford. She was also the niece of American poet Julia Ward Howe, and from her mother's remarriage (after the death of her father in 1857), she was the half-sister of Margaret Ward Terry, who later became the wife of Winthrop Astor Chanler. As Mrs. Hugh Fraser, Mary was the author of several memoirs and numerous works of fiction, eventually becoming better known than her husband. Together, they were the parents of two sons: