In 1760 he completed construction of his summer home and estate at "Georgia Plantation", southwest of Baltimore along the Georgetown Road, also known as the Columbia Road (later Washington Boulevard, (U.S. Route 1) in modern Carroll Park and north of the Gwynns Falls stream which flows into the Middle Branch ("Ridgeley's Cove") and Ferry Branch of the Patapsco River. He named the home "Mount Clare" after his grandmother. In June 1763 Charles married, to Margaret Tilghman (1742–1817), daughter of Matthew Tilghman of Talbot County. Although the couple had no children who reached maturity, they remained together until his death. She became the mistress of Mount Clare, and earned a reputation for her greenhouse and pinery, where she grew oranges, lemons, and pineapple.