1. As mentioned in Paper Summary, clear distinction between the 3 classes of Extreme Speech are needed. While the authors included definitions, I still find it difficult to differentiate derogatory extreme speech from exclusionary extreme speech. For instance, in the sample data file you provided, why is the instance "I support it 100/% #मुस्लिमो_का_संपूर्ण_बहिष्कार" exclusionary extreme speech but derogatory extreme speech? It seems that annotators took into account of local regulation over speech (line 438-441) - which regulation exactly?. Since the local regulation plays a role in annotations, does it reflect on zero-shot cross-country classification? - maybe the poor performance is due to annotation variance not model capability (line 546-548 v.s. line 553-558).
I understand that this task is complicated and believe that adding some examples will help. If possible, providing a screenshot or text description of annotation guideline will be great.
2. The dataset contains only extreme speech - it seems that the authors filtered out somehow the neutral text (or ones that don't require moderation according to their definitions). Why did you decide to discard them? Who set the criteria?
1. In Table 14, the α and overall accuracy are especially low for India. Is there any explanation?
2. In appendix, there are many tables with very little description or even none. If a table is added, it would be better to include relevant discussions. Otherwise, readers can only guess what it meant.
3. One interesting analysis would be comparing annotations between passages using only local language and the ones using local language and English (code-switching).
1. As mentioned in Paper Summary, clear distinction between the 3 classes of Extreme Speech are needed. While the authors included definitions, I still find it difficult to differentiate derogatory extreme speech from exclusionary extreme speech. For instance, in the sample data file you provided, why is the instance "I support it 100/% #मुस्लिमो_का_संपूर्ण_बहिष्कार" exclusionary extreme speech but derogatory extreme speech? It seems that annotators took into account of local regulation over speech (line 438-441) - which regulation exactly?. Since the local regulation plays a role in annotations, does it reflect on zero-shot cross-country classification?