The Mount Vernon Hotel operated until 1833, when it was purchased by Jeremiah Towle, who converted it to a private residence. His daughters continued to live in the house through 1905, when the Standard Gas Light Company bought the house and erected gas tanks nearby. The house was then bought by Jane Teller Robinson in 1919. The Colonial Dames of America purchased the site in 1924 to use as its headquarters. In 1939, the house opened to the public as the Abigail Adams Smith Museum. The planting plan for the Abigail Adams Smith Museum was by New York landscape designer Alice Recknagel Ireys.