While clearly there is some dissention in the ranks the ACLU has always stood up for the First Amendment even fighting cases for the KKK to hold parades and assemble peacefully.

The question here may be whether the white supremacists ever meant to be peaceful in Charlottesville.   The carried torches and chanted Nazi slogans while brandishing guns outside a synagogue.  

That's not peaceful.

They came looking for a fight just like they've done before.   And they came armed and they hurt people and one of them even murdered someone while injuring many others.

That's not peaceful assembly.

Would you defend Muslims if they did the same thing?