So, let me get this straight.  Rather than allow the person who identifies as gender y (despite being born as gender x) to use the bathroom (a private function, in a closed segment of a closed room) for people of gender y (which effectively puts them into a room full of people they are unlikely to be interested in), you would rather have the gender y identifying person (who by your reckoning is "abnormal" - FYI not my position, just that of Alaska Bigot Values) use the gender x bathroom (thereby putting the "abnormal" person in the same bathroom as you?

Got it.  I think I've found the flaw in your plan.