Zeidler faced the vexing issue of race relations as Milwaukee's African-American population tripled during the 1950s. Zeidler was a vocal supporter of the civil rights movement and his opponents tried to exploit this to their advantage. Zeidler's political enemies spread false rumors that Zeidler had put billboards in the South asking blacks to come north. Many workers in Milwaukee were threatened for supporting Zeidler. One manufacturer even threatened to fire employees who voted for Zeidler. Zeidler cited bad personal health and the race issue as reasons for not running for re-election in 1960.