At the time he was assigned to the Harpers Ferry Armory, he was 58 years old; the oldest colonel in the Union Army. Miles arrived to take command in the spring of 1862. In 1861 the armory had been burned by Confederates. The town's remaining houses, mills and churches served as hospitals, barracks and stables. Most of the population was gone. But it was still an important base of supply for the Union. By September 1862, Miles had 14,000 men at Harpers Ferry. On September 12, 1862 Brigadier general Julius White arrived. But because he had no military experience, Colonel Miles remained in command. On September 5, Miles had been given very specific orders by his superior, Major general John E. Wool, "you will not abandon Harpers Ferry without defending it to the last extremity." As if the message was not specific enough, Wool sent another saying "there must be no abandoning of a post, and shoot the first man that thinks of it". Miles wired back saying he was ready for them. But for some reason Miles insisted on keeping most of the troops near the town instead of taking up commanding positions on the surrounding heights. 