Where the practice is followed, women are typically forced to wear it.  Also, men don't have the same obligation to do the same, despite there being no reason for them to be excluded (you know, in order to be modest and virtuous like their women).  Therefore, it is the practice itself that is intolerable.  

You're being a cultural relativist.  I'm simply not tolerating what is clearly intolerable.