In 1864 they started building the Asylum to hold the prisoners who had become insane. During the 1860s and 1870s the prisoners left at the prison were either too old, too sick, or insane to keep working. The prison closed in 1877.