In 1835, Garnet enrolled at the new Noyes Academy in Canaan, New Hampshire, but anti-abolitionists soon destroyed the school building and forced the Negro students out of town. He completed his education at the Oneida Institute in Whitesboro, New York, which had recently begun admitting all races. Here he was acclaimed for his wit, brilliance, and rhetorical skills. The year after graduation in 1839, he injured his knee playing sports. It never recovered, and his lower leg had to be amputated in 1841.