suddenly rats appeared from everywhere throughout the city , thousands of them , and , when the pied piper began walking towards the gates of the city , the rats followed behind as if they had been trained to march in line .