The situation of the black students in California is not grounded in threats and not even objectively reasonable grounds to believe they are being threatened.  It's grounded in speech, pictures and actions motivated by bigotry and racial hatred.  The problem is that those speech, pictures and actions create a toxic atmosphere.  Nobody deserves to work or go to school under such conditions.  Living surrounded by hatred and contempt is not theoretical, it's real and it's bad.  

Your argument sets up a straw man of feelings of fear neither reasonable nor based in bigotry/racism/misogyny.  Of course there will never be a cause of action, or grounds for a complaint based on the fear of little people cowed by the mere presence of big people.

Up to you of course, but I wish you'd reconsider writing such things as "nobody has a right "not to feel" anything."  It only feeds anti-human rights bigots.