You say there are so many exceptions where it would be acceptable for white people to use the n word. I would argue that most folks writing about the history of racism would not use that word except to reference a primary source. 

The word was weaponozed long ago by white slave owners when they used it to refer to  black people as property. Historically the word preceeded a slave's "given" name so they would never be mistaken for a white person with that same name in conversation. Or in case, you know, somebody ever forgot that they were a slave.

Just because a group of people has reclaimed a word used to subjugate and dehumanize doesn't mean you have the right to use it too. If you are not a member of the opressed group and you use an abusive slur, you are a serious jerk and should be called out as such. Freedom of expression doesn't guarantee  you freedom from ridicule.