Lee was dumbfounded when the two staff officers he had sent with orders for Scott returned with the news that he was nowhere to be found and disconcerted by their reports of the British returning in force. When he observed part of Lafayette's force retreating after a failed attempt to silence some British artillery, it appeared to Lee that the right flank too was pulling back without orders. It had become clear that he was losing control of the vanguard, and with his immediate command now only 2,500 strong, he realized his plan to envelop the British rearguard was finished. His priority now was the safety of his command in the face of superior numbers.