But the fact is, white supremacists, neo-Nazi's, KKK, whatever they want to call themselves today, have been holding demonstrations for years. Decades. And people either ignored them, (which is the best punishment, IMO), or stared them down silently, or turned their back like you describe. And police made sure nothing got out of hand, & dispersed them when it looked like it might.  

Had the anti-Nazi folks peacefully demonstrated, & police kept them apart as is normal, the violence wouldn't have happened.  Someone has to start & respond to violence for violence to take place, & the police have to stand down, which they only do when told to. Why were they told to? Might the mentally deranged man who drove his care into the crowd have done so if it stayed non-violent and the police dispersed both crowds? Possibly.  No one knows.  But violent demonstrations and violent responses to demonstrations are a civil rights/criminal issue, not okay because most hate what one side stands for.