In November 1784, former Lieutenant Colonel (later Major General, Senator, and Governor of Virginia) Henry "Light Horse Harry" Lee III (1756–1818) purchased three one-half acre lots in Alexandria from Baldwin Dade (1716–1783), a merchant. On December 4, 1784, Lee sold one of these tracts to his father-in-law Philip Richard Fendall I, Esq. (1734–1805), for three hundred pounds. Fendall, using enslaved laborers, began building the Lee–Fendall House (for much of its life called the Phillip Fendall House), for his second wife, Elizabeth (Steptoe) Lee (1743–1789), in the spring or early summer of 1785. The lot was located on the Southeast corner of Washington and Oronoco Street, then the edge of the city. At the time, very few structures were near, and the Fendalls had uninterrupted views of Oronoco Bay and the ships which docked there. To the north and west lay verdant fields of grass and clover. Alexandria was an up-and-coming social and political center in Northern Virginia. The architect is unknown, but the style is similar to that found at "Hard Bargain", an estate built by the Digges family, and located in Charles County, Maryland, from which Fendall hailed. It consisted of a "telescopic" design, which was synonymous with Maryland, and had three sections. A plat on a 1796 insurance policy shows eight buildings on the quarter block, valued at a total of $11,500, including a "Rabbit House" and a "Pigeon House". The main dwelling house was valued at $5,000.