You have it backwards.

If all Muslims agreed this was discrimination against Muslims, they might have a point, although not necessarily.

When a substantial number of Muslims are for a ban which applies only to extremist Muslims, you cannot argue the ban is "against Muslims" in general.