Update README.md
Browse files
README.md
CHANGED
@@ -9,16 +9,16 @@ metrics:
|
|
9 |
pipeline_tag: zero-shot-classification
|
10 |
---
|
11 |
|
12 |
-
# XLM-ROBERTA-BASE-
|
13 |
|
14 |
## Model description
|
15 |
This model takes the XLM-Roberta-base model which has been continued to pre-traine on a large corpus of Twitter in multiple languages.
|
16 |
It was developed following a similar strategy as introduced as part of the [Tweet Eval](https://github.com/cardiffnlp/tweeteval) framework.
|
17 |
-
The model is further finetuned on the
|
18 |
|
19 |
## Intended Usage
|
20 |
|
21 |
-
This model was developed to do Zero-Shot Text Classification in the realm of Hate Speech Detection. It is focused on the language of spanish as it was finetuned on data in
|
22 |
|
23 |
### Usage with Zero-Shot Classification pipeline
|
24 |
```python
|
@@ -30,27 +30,26 @@ classifier = pipeline("zero-shot-classification",
|
|
30 |
After loading the model you can classify sequences in the languages mentioned above. You can specify your sequences and a matching hypothesis to be able to classify your proposed candidate labels.
|
31 |
|
32 |
```python
|
33 |
-
sequence_to_classify = "
|
34 |
|
35 |
|
36 |
# we can specify candidate labels and hypothesis:
|
37 |
-
candidate_labels = ["
|
38 |
-
hypothesis_template = "
|
39 |
|
40 |
# classify using the information provided
|
41 |
classifier(sequence_to_classify, candidate_labels, hypothesis_template=hypothesis_template)
|
42 |
|
43 |
|
44 |
# Output
|
45 |
-
#{'sequence': '
|
46 |
-
#'labels': ['
|
47 |
-
#'scores': [0.
|
48 |
-
|
49 |
```
|
50 |
|
51 |
|
52 |
## Training
|
53 |
-
This model was pre-trained on a set of 100 languages and follwed further training on 198M multilingual tweets as described in the original [paper](https://arxiv.org/abs/2104.12250). Further it was trained on the training set of XNLI dataset in
|
54 |
- learning rate: 5e-5
|
55 |
- batch size: 32
|
56 |
- max sequence: length 128
|
@@ -62,5 +61,5 @@ on one GPU (NVIDIA GeForce RTX 3090) resulting in a training time of 1h 47 mins.
|
|
62 |
The model was evaluated after each epoch on the eval set of the XNLI Corpus and at the end of training on the Test set of the XNLI corpus.
|
63 |
Using the test set the model reached an accuracy of
|
64 |
```
|
65 |
-
predict_accuracy =
|
66 |
```
|
|
|
9 |
pipeline_tag: zero-shot-classification
|
10 |
---
|
11 |
|
12 |
+
# XLM-ROBERTA-BASE-XNLI-ES
|
13 |
|
14 |
## Model description
|
15 |
This model takes the XLM-Roberta-base model which has been continued to pre-traine on a large corpus of Twitter in multiple languages.
|
16 |
It was developed following a similar strategy as introduced as part of the [Tweet Eval](https://github.com/cardiffnlp/tweeteval) framework.
|
17 |
+
The model is further finetuned on the spanish part of the XNLI training dataset.
|
18 |
|
19 |
## Intended Usage
|
20 |
|
21 |
+
This model was developed to do Zero-Shot Text Classification in the realm of Hate Speech Detection. It is focused on the language of spanish as it was finetuned on data in said language. Since the base model was pre-trained on 100 different languages it has shown some effectiveness in other languages. Please refer to the list of languages in the [XLM Roberta paper](https://arxiv.org/abs/1911.02116)
|
22 |
|
23 |
### Usage with Zero-Shot Classification pipeline
|
24 |
```python
|
|
|
30 |
After loading the model you can classify sequences in the languages mentioned above. You can specify your sequences and a matching hypothesis to be able to classify your proposed candidate labels.
|
31 |
|
32 |
```python
|
33 |
+
sequence_to_classify = "Creo que Lionel Messi es el mejor futbolista del mundo."
|
34 |
|
35 |
|
36 |
# we can specify candidate labels and hypothesis:
|
37 |
+
candidate_labels = ["politíca", "futbol"]
|
38 |
+
hypothesis_template = "Este ejemplo es {}"
|
39 |
|
40 |
# classify using the information provided
|
41 |
classifier(sequence_to_classify, candidate_labels, hypothesis_template=hypothesis_template)
|
42 |
|
43 |
|
44 |
# Output
|
45 |
+
#{'sequence': 'Creo que Lionel Messi es el mejor futbolista del mundo.',
|
46 |
+
# 'labels': ['futbol', 'politíca'],
|
47 |
+
# 'scores': [0.813454806804657, 0.18654517829418182]}
|
|
|
48 |
```
|
49 |
|
50 |
|
51 |
## Training
|
52 |
+
This model was pre-trained on a set of 100 languages and follwed further training on 198M multilingual tweets as described in the original [paper](https://arxiv.org/abs/2104.12250). Further it was trained on the training set of XNLI dataset in spanish which is a machine translated version of the MNLI dataset. It was trained on 3 epochs and the following specifications
|
53 |
- learning rate: 5e-5
|
54 |
- batch size: 32
|
55 |
- max sequence: length 128
|
|
|
61 |
The model was evaluated after each epoch on the eval set of the XNLI Corpus and at the end of training on the Test set of the XNLI corpus.
|
62 |
Using the test set the model reached an accuracy of
|
63 |
```
|
64 |
+
predict_accuracy = 78.82 %
|
65 |
```
|