The property that became Patchin Place was once part of a farm belonging to Sir Peter Warren. In 1799 it was sold to Samuel Milligan, who later conveyed it to his son-in-law, Aaron Patchin. The buildings that now occupy the site were put up in 1848 or 1849. Many guide books say the buildings were intended to be boarding houses for Basque waiters and other workers at the Brevoort House hotel on Fifth Avenue, but the Brevoort was not built until 1855. The rooms were small, and at the time, the street was noisy due to its proximity to the vendors in Jefferson Market.