The marshy site on which the prison stood meant that the builders experienced problems of subsidence from the outset, and explains the succession of architects. it was to consist of a hexagonal central courtyard with an elongated pentagonal courtyard on each outer wall of the central courtyard; the three outer corners of the pentagonal courtyards each had a tower one storey higher than the three floors of the rest of the building. When the boundary wall reached a height of about six feet high it began to tilt and crack. After 18 months, with £26,000 spent on it, Hardwick resigned in 1813, and John Harvey took over the role. Harvey was dismissed in turn in 1815, and replaced by Robert Smirke, who brought the project to completion in 1821. Further legislation was passed in the form of the Millbank Penitentiary Act of 1816 (56 Geo. 3 c. 63).