Col. Tayloe had been considering building his new home in Philadelphia under the expert hands of architect Benjamin Latrobe. Instead he chose the primitive wilds of the new federal city and the architect Dr. William Thornton, the man who designed the new United States Capitol and the help of James Hoban winner of the White House design contest. Col. Tayloe went with the wishes of George Washington.
Col. Tayloe had considered building his new home in Philadelphia with Latrobe’s guidance. However, he chose to build in the untamed wilderness of the federal city and employed Dr. William Thornton, who designed the United States Capitol and won the White House design contest. Col. Tayloe followed George Washington's wishes.