On October 2, 1795, at the age of 19, Mary married John Pickersgill, a merchant, and moved back to Philadelphia with her husband. Of Mary's four children, only one survived childhood, a daughter named Caroline. Mary's husband traveled to London to work for the United States Government in the British Claims Office, but died in London on June 14, 1805, leaving Mary widowed at the age of 29. In 1807 Mary then moved back to Baltimore with her daughter Caroline and her 67-year-old mother Rebecca.