I think that Neil Degrasse-Tyson had the best response to this kind of argument. He pointed out that when he was growing up, literally no one in his school or larger learning environment encouraged his interest in astrophysics. Instead, because he was black, his teachers kept pushing him towards sports. He was telling this story in response to a question about the lack of women in the sciences. The upshot of his response was to say that before people began making categorical statements about how people are biologically different, we need to know that social and environmental factors are really controlled for and are not decisively at play. That argument continues to apply here. It may be true that different kinds of biological factors affect the disposition and interests of men and women in general terms. However, we need to construct an environment that allows for individuals to make their own choices. Generalizing about groups of people discourages those exceptional individuals.