The overt racism of the white supremacists is part of the problem, but you can also see it in police behaviour, so-called "racial profiling." The beating of Rodney King was an extreme example of this practice.

Then there's covert racism that dooms many blacks to lives of poverty.

So all in all, the white supremacists are a pretty small part of the picture.