''Think about it'' 

Think about it some more.

Your comment is a variation of what is sometimes called the Fallacy of Pathos. i.e. "that one's emotions, urges or "feelings" are innate and in every case self-validating, autonomous, and above any human intent or act of will (one's own or others), and are thus immune to challenge or critique".

No one is dismissing any of what you point out. But it cannot be rationally used as a justification. To react to difficult circumstances does not require a racist or bigoted response. That is a separate choice that an individual makes. And is often made without full knowledge or grasp of the facts - poor education, poverty and wilful ignorance, being the shaky foundation upon which many of  these choices are made.