more than half of the middle bailey had been impassable from the wall what much and john brought down , and the bailiff , the only person left to run the castle , moved his quarters to the lowermost bailey .