Disagree.  I would say most people don't give a flying fig... whether you're trans or not, gay or not, rich or not, black white green or not... but "not caring" is more in the vein of "An it harm none do what ye will" as opposed to "it's weird and I'm gonna pretend it doesn't exist".

To the OP: much empathy.  It must be hard enough being a member of a truly rare and misunderstood community without having band wagoners jump on board, diluting and distorting a very real issue for their own narcissistic and myopic views.