During the administration of President Ulysses S. Grant (1869-1877), Brentwood Mansion was a center of Washington social life, with many parties hosted by Elizabeth (Eliza) Worthington Patterson (daughter of Joseph Pearson and his third wife Catherine Worthington Pearson) and her husband Carlile Pollock Patterson. Patterson had known Grant since 1852; Patterson's brother-in-law David Dixon Porter, married to Patterson's sister George Ann, was the highest-ranking Admiral in the US Navy; and Patterson's brother Thomas H. Patterson was commander of the Washington Navy Yard (1873-1876) before becoming the Rear Admiral in command of the Asiatic Squadron (1877-1880). Patterson Street, N.E., which is adjacent to the property, was probably named for the Patterson family.