Mary and Robert were married at her parents' home, Arlington House, on 30 June 1831. They had three sons and four daughters together: George Washington Custis "Custis", William H. Fitzhugh "Rooney", Robert Edward Jr., Mary, Eleanor Agnes (called Agnes), Anne, and Mildred Lee.