|
--- |
|
license: mit |
|
language: |
|
- la |
|
- el |
|
- fr |
|
- en |
|
- de |
|
- it |
|
base_model: |
|
- FacebookAI/xlm-roberta-base |
|
--- |
|
|
|
# Model Description |
|
|
|
<!-- Provide a quick summary of what the model is/does. --> |
|
|
|
This model checkpoint was created by further pre-training XLM-RoBERTa-base on a 1.4B tokens corpus of classical texts mainly written in Ancient Greek, Latin, French, German, English and Italian. |
|
The corpus notably contains data from [Brill-KIEM](https://github.com/kiem-group/pdfParser), various ancient sources from the Internet Archive, the [Corpus Thomisticum](https://www.corpusthomisticum.org/), [Open Greek and Latin](https://www.opengreekandlatin.org/), [JSTOR](https://about.jstor.org/whats-in-jstor/text-mining-support/), [Persée](https://www.persee.fr/), Propylaeum, [Remacle](https://remacle.org/) or Wikipedia. |
|
The model can be used as a checkpoint for further pre-training or as a base model for fine-tuning. |
|
The model was evaluated on classics-related named-entity recognition and part-of-speech tagging and surpassed XLM-RoBERTa-Base on all task. |
|
It also performed significantly better than similar models retrained from scratch on the same corpus. |
|
|
