You've gotten your facts wrong, Navin ( FYI, the counter protesters were legally permitted; it's the racist groups who were unpermitted and/or violated the terms of their permit and had it taken away).

 But more importantly, you've missed the point.  This isn't about free speech, any more the reason the White Supremacists, KKK and Nazis  were in Charlotte to protest taking down a statue if Robert E. Lee.  And as others have pointed out, there are certain restrictions on free speech. Those restrictions include threatening to kill someone when there is evidence of intent, which there certainly was both the night before and the day of the riot.  Then there's whole "yelling fire in a crowded theatre" exception which says deliberating creating a dangerous situation where people are likely to get hurt - say, inciting a riot - is not protected speech.  

Words have consequences as much as actions do.