---
language: 
  - en
library_name: pysentimiento

tags:
  - twitter
  - irony
---

# Irony detection in English
## bertweet-irony

Repository: [https://github.com/pysentimiento/pysentimiento/](https://github.com/finiteautomata/pysentimiento/)



Model trained with SemEval 2018 dataset Task 3 (Van Hee et all, 2018) for irony detection. Base model is [BERTweet], a RoBERTa model trained in English tweets.

The positive class marks irony, the negative class marks not ironic content.

## Results

Results for the four tasks evaluated in `pysentimiento`. Results are expressed as Macro F1 scores



| Model      | sentiment   | emotion     | hate_speech   | irony       |
|:-----------|:------------|:------------|:--------------|:------------|
| bert       | 69.6 +- 0.4 | 42.7 +- 0.6 | 56.0 +- 0.8   | 68.1 +- 2.2 |
| electra    | 70.9 +- 0.4 | 37.2 +- 2.9 | 55.6 +- 0.6   | 71.3 +- 1.8 |
| roberta    | 70.4 +- 0.3 | 45.0 +- 0.9 | 55.1 +- 0.4   | 70.4 +- 2.9 |
| robertuito | 69.6 +- 0.5 | 43.0 +- 3.3 | 57.5 +- 0.2   | 73.9 +- 1.4 |
| bertweet   | 72.0 +- 0.4 | 43.1 +- 1.8 | 57.7 +- 0.7   | 80.8 +- 0.7 |


Note that for Hate Speech, these are the results for Semeval 2019, Task 5 Subtask B (HS+TR+AG detection)

## Citation

If you use this model in your research, please cite pysentimiento, dataset and pre-trained model papers:

```
@misc{perez2021pysentimiento,
      title={pysentimiento: A Python Toolkit for Sentiment Analysis and SocialNLP tasks},
      author={Juan Manuel Pérez and Juan Carlos Giudici and Franco Luque},
      year={2021},
      eprint={2106.09462},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}
@inproceedings{van2018semeval,
  title={Semeval-2018 task 3: Irony detection in english tweets},
  author={Van Hee, Cynthia and Lefever, Els and Hoste, V{\'e}ronique},
  booktitle={Proceedings of The 12th International Workshop on Semantic Evaluation},
  pages={39--50},
  year={2018}
}

@inproceedings{nguyen2020bertweet,
  title={BERTweet: A pre-trained language model for English Tweets},
  author={Nguyen, Dat Quoc and Vu, Thanh and Nguyen, Anh Tuan},
  booktitle={Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations},
  pages={9--14},
  year={2020}
}
```