ezio realized that as most of the medici defenders would be concentrated at the front of the building where most of the fighting was , they probably did n't have enough men to cover the rear as well .