The house is named for James Pitot, the fourth owner of the house who resided there from 1810 to 1819. Pitot is considered to be the first "American" mayor of New Orleans (1804–1805); for although he was a native of France, he became a naturalized American citizen before arriving in New Orleans in 1796. Inside are American and Louisiana antiques from the early 19th century, but the antiques are not original to the home. A portrait of Sophie Gabrielle, James Pitot's daughter, is the only artifact owned by any past resident of the house.