Free speech has its limits. People should not be allowed to say hateful things about other groups without repercussions. 
That's why civil societies developed, and greatly reduced the hateful speech towards Jews and Blacks.