According to journalists interviewing them, the people who descended on Charlottesville call themselves white supremacists, Nazis, and fascists.  

So why would you even think that it's a wild accusation to call them what they are?  

Give your head a shake.