they were in mayeyes fortress , actually the city surrounding the castle but since the place had grown so large and the population so vast the perimeter was refortified with a wall around the complexes outside the palace proper .