File size: 2,258 Bytes
495e076 83492c3 495e076 b643a7e 06ed9a5 b643a7e 495e076 b643a7e |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 |
---
language:
- en
library_name: pysentimiento
tags:
- twitter
- irony
---
# Irony detection in English
## bertweet-irony
Repository: [https://github.com/pysentimiento/pysentimiento/](https://github.com/finiteautomata/pysentimiento/)
Model trained with SemEval 2018 dataset Task 3 (Van Hee et all, 2018) for irony detection. Base model is [BERTweet], a RoBERTa model trained in English tweets.
The positive class marks irony, the negative class marks not ironic content.
## Results
Results for the four tasks evaluated in `pysentimiento`. Results are expressed as Macro F1 scores
| Model | sentiment | emotion | hate_speech | irony |
|:-----------|:------------|:------------|:--------------|:------------|
| bert | 69.6 +- 0.4 | 42.7 +- 0.6 | 56.0 +- 0.8 | 68.1 +- 2.2 |
| electra | 70.9 +- 0.4 | 37.2 +- 2.9 | 55.6 +- 0.6 | 71.3 +- 1.8 |
| roberta | 70.4 +- 0.3 | 45.0 +- 0.9 | 55.1 +- 0.4 | 70.4 +- 2.9 |
| robertuito | 69.6 +- 0.5 | 43.0 +- 3.3 | 57.5 +- 0.2 | 73.9 +- 1.4 |
| bertweet | 72.0 +- 0.4 | 43.1 +- 1.8 | 57.7 +- 0.7 | 80.8 +- 0.7 |
Note that for Hate Speech, these are the results for Semeval 2019, Task 5 Subtask B (HS+TR+AG detection)
## Citation
If you use this model in your research, please cite pysentimiento, dataset and pre-trained model papers:
```
@misc{perez2021pysentimiento,
title={pysentimiento: A Python Toolkit for Sentiment Analysis and SocialNLP tasks},
author={Juan Manuel Pérez and Juan Carlos Giudici and Franco Luque},
year={2021},
eprint={2106.09462},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
@inproceedings{van2018semeval,
title={Semeval-2018 task 3: Irony detection in english tweets},
author={Van Hee, Cynthia and Lefever, Els and Hoste, V{\'e}ronique},
booktitle={Proceedings of The 12th International Workshop on Semantic Evaluation},
pages={39--50},
year={2018}
}
@inproceedings{nguyen2020bertweet,
title={BERTweet: A pre-trained language model for English Tweets},
author={Nguyen, Dat Quoc and Vu, Thanh and Nguyen, Anh Tuan},
booktitle={Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations},
pages={9--14},
year={2020}
}
``` |