Harmon was preceded by his two older brothers, Kenneth B. Harmon, of the class of 1910, and Millard F. Harmon, Jr., of the class of 1912. Hubert Harmon was ejected when it was discovered that he had two brothers at West Point, as it was felt that the taxpayers would not be getting their money's worth educating three sons of the same family. He was readmitted in June 1911 after his father lobbied numerous government officials. He now only had one brother at West Point, as Kenneth had graduated the year before. Both Millard Jr., and Hubert subsequently became three-star generals.