|
--- |
|
library_name: xpmir |
|
--- |
|
# monoT5 trained on MS-Marco |
|
Implementation of |
|
|
|
Nogueira, R., Jiang, Z., Lin, J., 2020. Document Ranking with a Pretrained Sequence-to-Sequence Model. arXiv:2003.06713 [cs]. |
|
|
|
This model has been trained on MsMarco v1, and uses the t5-base model |
|
|
|
Parameters based on [PyGaggle](https://raw.githubusercontent.com/vjeronymo2/pygaggle/master/pygaggle/run/finetune_monot5.py) |
|
|
|
|
|
|
|
|
|
|
|
## Using the model |
|
The model can be loaded with [experimaestro |
|
IR](https://experimaestro-ir.readthedocs.io/en/latest/) |
|
|
|
If you want to use the model in further experiments with XPMIR, |
|
use this code: |
|
```py |
|
from xpmir.models import AutoModel |
|
from xpmir.models import AutoModel |
|
|
|
model, init_tasks = AutoModel.load_from_hf_hub("xpmir/monot5") |
|
``` |
|
|
|
|
|
Use this code if you want to use the model in inference only: |
|
|
|
```py |
|
from xpmir.models import AutoModel |
|
from xpmir.models import AutoModel |
|
|
|
model = AutoModel.load_from_hf_hub("xpmir/monot5", as_instance=True) |
|
model.rsv("walgreens store sales average", "The average Walgreens salary ranges...") |
|
``` |
|
|
|
## Results |
|
|
|
| Dataset | AP | P@20 | RR | RR@10 | Success@5 | nDCG | nDCG@10 | nDCG@20 | |
|
|----| ---|------|------|------|------|------|------|------| |
|
| msmarco_dev | 0.3797 | 0.0384 | 0.3851 | 0.3762 | 0.5497 | 0.4835 | 0.4382 | 0.4602 | |
|
| trec2019 | 0.4874 | 0.7209 | 0.9671 | 0.9671 | 1.0000 | 0.6918 | 0.7217 | 0.6939 | |
|
| trec2020 | 0.4605 | 0.6139 | 0.9396 | 0.9389 | 0.9815 | 0.6796 | 0.6925 | 0.6581 | |
|
|
