The main part of the house was built in the 1740s, and an addition was made to it in the 1820s. A legend states that during the American Revolutionary War, General George Washington and the Marquis de Lafayette talked about war strategy under an apple tree in front of the house in 1779. However, this is probably not true. There used to be an apple orchard and a cider press at the house, which is why it is also called the Apple Tree House.