In 1818–19, prior to his appointment as provisional Governor of the Florida Territory, Jackson built a brick house to replace the log structure he had lived in after purchasing the land. He also added six brick structures, containing a total of 13 dwelling units for the people he enslaved. These included a three-unit building known as the Triplex, built behind the mansion. A two-unit duplex known as the South Cabin was built at the First Hermitage. Four brick duplexes were built at the Field Quarter and were known as Cabins 1, 2, 3, and 4. At least one Field Cabin was still standing as late as the 1920s.