So are you going to have someone checking peoples birth certificates and equipment whenever they enter a restroom?
You're right, this should not be an issue. There have always been transgendered people using the restroom of their choice, suddenly it's something to be afraid of.