add comparison details
Browse files
README.md
CHANGED
@@ -119,7 +119,7 @@ It can be used for [Extractive Q&A](https://huggingface.co/tasks/question-answer
|
|
119 |
|
120 |
[Evaluation notebook](https://github.com/anakin87/electra-italian-xxl-cased-squad-it/blob/main/evaluation.ipynb)
|
121 |
|
122 |
-
## Usage in Transformers
|
123 |
Model checkpoints are available for usage in Pytorch. They can be used directly with pipelines as:
|
124 |
|
125 |
```python
|
@@ -129,11 +129,28 @@ qa = pipeline('question-answering', model='anakin87/electra-italian-xxl-cased-sq
|
|
129 |
qa(question="Qual è il soprannome di Vasco Rossi?", context="Vasco Rossi, noto anche semplicemente come Vasco e in passato con l'appellativo Blasco (Zocca, 7 febbraio 1952), è un cantautore italiano")
|
130 |
>>> {'score': 0.93, 'start': 80, 'end': 86, 'answer': 'Blasco'}
|
131 |
```
|
132 |
-
## Usage in Haystack
|
133 |
|
134 |
TO DO: link notebook
|
135 |
|
136 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
137 |
[Training notebook](https://github.com/anakin87/electra-italian-xxl-cased-squad-it/blob/main/training.ipynb)
|
138 |
|
139 |
**Hyperparameters**
|
|
|
119 |
|
120 |
[Evaluation notebook](https://github.com/anakin87/electra-italian-xxl-cased-squad-it/blob/main/evaluation.ipynb)
|
121 |
|
122 |
+
## Usage in Transformers 🚀
|
123 |
Model checkpoints are available for usage in Pytorch. They can be used directly with pipelines as:
|
124 |
|
125 |
```python
|
|
|
129 |
qa(question="Qual è il soprannome di Vasco Rossi?", context="Vasco Rossi, noto anche semplicemente come Vasco e in passato con l'appellativo Blasco (Zocca, 7 febbraio 1952), è un cantautore italiano")
|
130 |
>>> {'score': 0.93, 'start': 80, 'end': 86, 'answer': 'Blasco'}
|
131 |
```
|
132 |
+
## Usage in Haystack 🚀🚀🚀
|
133 |
|
134 |
TO DO: link notebook
|
135 |
|
136 |
+
## Comparison ⚖️
|
137 |
+
|
138 |
+
| Model | EM | F1 | Model size (pytorch) |
|
139 |
+
|------------------------------------------------------------|-------|-------|----------------------|
|
140 |
+
| it5/it5-large-question-answering | 69.10 | 78.00 | 3.13 GB |
|
141 |
+
| **anakin87/electra-italian-xxl-cased-squad-it (this one)** | 66.03 | 77.47 | 437 MB |
|
142 |
+
| it5/it5-base-question-answering | 66.30 | 76.10 | 990 MB |
|
143 |
+
| it5/mt5-base-question-answering | 66.30 | 75.70 | 2.33 GB |
|
144 |
+
| antoniocappiello/bert-base-italian-uncased-squad-it | 63.80 | 75.30 | 440 MB |
|
145 |
+
| luigisaetta/squad_it_xxl_cased_hub1 | 63.95 | 75.27 | 440 MB |
|
146 |
+
| it5/it5-efficient-small-el32-question-answering | 64.50 | 74.70 | 569 MB |
|
147 |
+
| mrm8488/bert-italian-finedtuned-squadv1-it-alfa | 62.51 | 74.16 | 440 MB |
|
148 |
+
| mrm8488/umberto-wikipedia-uncased-v1-finetuned-squadv1-it | 60.50 | 72.41 | 443 MB |
|
149 |
+
| it5/it5-small-question-answering | 61.90 | 71.60 | 308 MB |
|
150 |
+
| it5/mt5-small-question-answering | 56.00 | 66.00 | 1.2 GB |
|
151 |
+
| DrQA-it trained on SQuAD-it | 56.10 | 65.90 | ? |
|
152 |
+
|
153 |
+
## Training details 🏋️
|
154 |
[Training notebook](https://github.com/anakin87/electra-italian-xxl-cased-squad-it/blob/main/training.ipynb)
|
155 |
|
156 |
**Hyperparameters**
|