I think you are confusing truth, fact and inference.
A statement of fact is the result of a direct observation.
In a statement of inference we combine facts  with our past experiences, with things we have read or heard, and our own personal bias, to come up with a statement of inference.
There is nothing wrong with an inference, but it is a problem when you treat the inference as a fact, and refer to it as "truth".
*
If you have seen a white person dissing some other person who is not white because of their ethnic background, then that is a fact.
When you make broad generalizations using fabricated categories then it is an inference, and treating it as truth is not clear thinking.
It is the source of many of the world's problems.