The original fieldstone section of the house was erected in about 1790 for Richard Sloan and his wife Charlotte VanHorn Sloan. Originally from Ireland, Sloan arrived in the United States after the American Revolutionary War and became an indentured servant of David VanHorn. Sloan eloped with VanHorn's daughter Charlotte and they settled in the Mill Creek valley, where they built the original stone portion of the house. The Sloans had ten children, including John and Thomas Sloan, who each (later) represented Hampshire County in the Virginia House of Delegates. Richard Sloan and his family operated a successful weaving business from the stone house and their Sloan counterpanes (woven coverlets with block designs) became well known in the South Branch Valley region.