Upon hearing the firing, Percy ordered his troops to continue their advance. British artillery fire forced Graydon in the first defensive line to fall back to the second line, where Washington, Greene, Putnam and Hugh Mercer were located. The four were encouraged to leave Manhattan, which they did immediately, sailing across the river to Fort Lee. Magaw realized that Cadwalader was in danger of being surrounded and sent orders for him to withdraw toward the fort. Cadwalader's force was pursued by Percy's troops at the same time the troops opposing Stirling's landing were also being chased back to the fort. Stirling's troops, landed in the rear of Cadwalader, paused, believing that there were troops in the entrenchments. Some of the retreating Americans engaged Stirling, giving most of the rest of the American troops enough time to escape.