What does "baseline" have to do with it? Men average taller than women; all you need to do to verify this is measure the male and female populations and do the simple averaging calculation. The same applies in any case of a difference between two populations; you simply compare the two populations; comparing both of them to a third "baseline" population makes no sense.