You would lose that bet. It is difficult to make a principled distinction between wanting a doctor of your own gender and a doctor of your own race or ethnicity, but most people will insist that there is a real difference and that the preference for a doctor of your own gender is legitimate. 

In fact, my own feeling (I'm male) is that I prefer a female doctor for very personal examinations. Intellectually, I know there's nothing sexual about a doctor examining my private parts, but having a male doctor do so feels vaguely homosexual (which I am not), and uncomfortable.