In early 1925, the U.S. Court of Appeal upheld the original court decision. Garvey was in Detroit at the time and was arrested while aboard a train back to New York City. In February he was taken to the Atlanta Federal Penitentiary and incarcerated there. Imprisoned, he was made to carry out cleaning tasks. On one occasion he was reprimanded for insolence towards the white prison officers. There, he became increasingly ill with chronic bronchitis and lung infections. Two years into his imprisonment he would be hospitalized with influenza.