it was only a matter of knocking over their ladders , and sending them falling the short distance to the ground , but when there were too many to handle , the ramparts were abandoned , and men started spilling into the trenches .