What percentage of the population are actually  Klansmen or Nazis'?  Probably not near enough to upset the entire country.  

Also there also a lot of "white men" in the Bernie camp, Obama camp, and democratic party.