It was clearly a serious error in judgement for any judge to wear anything that makes a political statement or which gives any signal of a lack of impartiality. 
At the same time the existence of such religious and racial based groups  as the Canadian Muslim Lawyers Association and the Association of Canadian Muslim women in law is also disturbing - special interest groups have no place in our legal system. The same standards of impartiality need to be applied.