Around noon on May 24, Lee's men charged and drove in the Union pickets who were posted near the Charles City Road, about a north of the fort. By 1:30 p.m. the fort was invested and Lee sent two officers under a flag of truce with a message demanding the surrender of the garrison. He promised that the black soldiers would be taken to Richmond and treated as prisoners of war, but if they did not surrender, he would not be "answerable for the consequences." Wild and his men interpreted this to mean that some of the men would be returned to their former masters and others would be tried by state authorities for inciting insurrection. Wild sent back a written reply that said "We will try it" and verbally told the two officers "Take the fort if you can."