Good rationalization and clarification, Chris.  I never thought of it that way.

Your line of thinking can be applied to other areas

For example, George Wallace wasn't against ALL blacks from attending that school, just those ones.

When feminists complain that a given woman was not elected, it's not like they were against ALL women, just that one.

Etc, etc, etc...