Update README.md
Browse files
README.md
CHANGED
|
@@ -8,6 +8,8 @@ license: apache-2.0
|
|
| 8 |
|
| 9 |
InformBERT is pretrained model trained using variable masking strategy, where informative tokens are masked more frequently compared to other tokens. InformBERT outperforms random masking based pretrained models on the factual recall benchmark LAMA and extractive question answering benchmark SQuAD.
|
| 10 |
|
|
|
|
|
|
|
| 11 |
## How to use
|
| 12 |
```Python
|
| 13 |
from transformers import AutoTokenizer, AutoModel
|
|
|
|
| 8 |
|
| 9 |
InformBERT is pretrained model trained using variable masking strategy, where informative tokens are masked more frequently compared to other tokens. InformBERT outperforms random masking based pretrained models on the factual recall benchmark LAMA and extractive question answering benchmark SQuAD.
|
| 10 |
|
| 11 |
+
More detail: https://arxiv.org/abs/2210.11771
|
| 12 |
+
|
| 13 |
## How to use
|
| 14 |
```Python
|
| 15 |
from transformers import AutoTokenizer, AutoModel
|