the city itself , within the massive black walls and bars of the fortress , was shared by a multifarious population , and though they had been stirred together over the centuries , still they tended to settle in neighborhoods like with like , or near enough , and a caste system prevailed , based on aspect .