Edmund J. Lee was a lawyer, councilman, Mayor, Clerk of the Circuit Court, and vestryman of Christ Church. He was also a board member of the Alexandria Academy and active in other civic organizations. Despite his prominence, Edmund, like his brother Harry previously, encountered financial distress and was forced to first mortgage the house, then sell it at auction in 1833. In 1836, his son Edmund Jennings Lee II was able to buy the house. Edmund Sr. then moved from his home across the street at 428 Washington into the Lee–Fendall House. Sally died at Lee-Fendall in 1837. Edmund Sr. bought the house back from Edmund Jr. in 1839 and continued to live there until his death in 1843.