A two-storey extension was made to the south side towards the end of the 13th century, with four windows lighting the upstairs room. Work was also carried out to the interior of the upper room during the 13th century, when the stone fireplaces were installed. The embattled north front was added to the building around 1400. A survey of the town's guns in 1468 reported that the Bargate held two breach loader guns and a brass muzzle loader. It is not clear when the Bargate started being used as a prison but the first records of it date from 1439. In 1458 the prison was used to detain the Genoese population of the town as part of the response to a Genoese attack on an English trading expedition.