not to ruin a good time , lot said , looking upwards at the hidden door from which they had just descended , `` but i think we need to be a little more concerned about the large group of angry little men who pursue us .