The entire plan hung on their ability to keep their numbers unknown.
As long as everyone could stay quiet, their size would come as a surprise.
The were quite a small group and had to stay hidden so that none would be killed.