At times, 200–300 boys attended the academy. The Choctaw students were at the school in the period prior to the Indian Removal in the 1830s of the "Five Civilized Tribes", but they were under pressure in the Southeast from encroaching settlers. His uncle kept the school going into the late 1830s, after some peoples had been forcibly relocated to Indian Territory west of the Mississippi River. The young Johnson went on to study at St. Joseph's College, an academy in Bardstown, and graduated.