File size: 1,461 Bytes
e6437df |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 |
---
library_name: xpmir
---
# monoT5 trained on MS-Marco
Implementation of
Nogueira, R., Jiang, Z., Lin, J., 2020. Document Ranking with a Pretrained Sequence-to-Sequence Model. arXiv:2003.06713 [cs].
This model has been trained on MsMarco v1, and uses the t5-base model
Parameters based on [PyGaggle](https://raw.githubusercontent.com/vjeronymo2/pygaggle/master/pygaggle/run/finetune_monot5.py)
## Using the model
The model can be loaded with [experimaestro
IR](https://experimaestro-ir.readthedocs.io/en/latest/)
If you want to use the model in further experiments with XPMIR,
use this code:
```py
from xpmir.models import AutoModel
from xpmir.models import AutoModel
model, init_tasks = AutoModel.load_from_hf_hub("xpmir/monot5")
```
Use this code if you want to use the model in inference only:
```py
from xpmir.models import AutoModel
from xpmir.models import AutoModel
model = AutoModel.load_from_hf_hub("xpmir/monot5", as_instance=True)
model.rsv("walgreens store sales average", "The average Walgreens salary ranges...")
```
## Results
| Dataset | AP | P@20 | RR | RR@10 | Success@5 | nDCG | nDCG@10 | nDCG@20 |
|----| ---|------|------|------|------|------|------|------|
| msmarco_dev | 0.3797 | 0.0384 | 0.3851 | 0.3762 | 0.5497 | 0.4835 | 0.4382 | 0.4602 |
| trec2019 | 0.4874 | 0.7209 | 0.9671 | 0.9671 | 1.0000 | 0.6918 | 0.7217 | 0.6939 |
| trec2020 | 0.4605 | 0.6139 | 0.9396 | 0.9389 | 0.9815 | 0.6796 | 0.6925 | 0.6581 |
|