Middleton began work on the plantation's gardens in 1741. Determined to outshine his neighbors, who were laying out neat four-squared patterned parterres, Middleton employed an English gardener named Simms, of whom little is known. Basing his designs less upon his own trip to England, Middleton and Simms derived their inspiration from the engraved plans in the translation of the popular garden book by Dezallier d'Argenville, "The Theory and Practice of Gardening". Middleton and Simms extended a main axis centered on the main block of the house, expressed as a gravel carriageway leading through a greensward to the first of six shaped turf terraces with bowed centers. At the level of the river a pair of lakes (the "Butterfly Lakes") were excavated on either side of a turf causeway prolonging the axis. With the river beyond, the lakes were flanked by more water, a stream dammed to the right to form the long narrow Rice Mill Pond, and a levee extended to the left to enclose floodable rice fields.