In 1875, he opposed the transfer of troops out of the South, fearing the effect of such a move on the safety of blacks in the South. During consideration of a bill to reduce and restructure the United States Army, Smalls introduced an amendment that "Hereafter in the enlistment of men in the Army...no distinction whatsoever shall be made on account of race or color." However, the amendment was not considered by Congress. He was the last Republican elected from the 5th district until 2010 when Mick Mulvaney took office. He was the second-longest serving African-American member of Congress (behind his contemporary Joseph Rainey) until the mid-20th century.