He married Florence O'Callaghan on 15 July 1869 at St. Marylebone, London, but his personal life took second place to his professional life. On opening night of "The Bells", 25 November 1871, Florence, who was pregnant with their second child, criticised his profession: "Are you going on making a fool of yourself like this all your life?" Irving exited their carriage at Hyde Park Corner, walked off into the night, and chose never to see her again. He maintained a discreet distance from his children as well, but became closer to them as they grew older. Florence Irving never divorced Irving, and once he had been knighted she styled herself "Lady Irving"; Irving never remarried.