File size: 1,420 Bytes
7d29d04 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 |
---
license: apache-2.0
datasets:
- unicamp-dl/mmarco
language:
- vi
library_name: sentence-transformers
pipeline_tag: sentence-similarity
tags:
- cross-encoder
- rerank
---
## Installation
- Install `sentence-transformers` (recommend):
- `pip install sentence-transformers`
- Install `transformers` (optional):
- `pip install transformers`
- Install `pyvi` to word segment:
- `pip install pyvi`
## Usage with transformers
```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch
model = AutoModelForSequenceClassification.from_pretrained('itdainb/vietnamese-cross-encoder')
tokenizer = AutoTokenizer.from_pretrained('itdainb/vietnamese-cross-encoder')
features = tokenizer(['How many people live in Berlin?', 'How many people live in Berlin?'], padding=True, truncation=True, return_tensors="pt")
model.eval()
with torch.no_grad():
scores = model(**features).logits
print(scores)
```
## Usage with sentence-transformers
Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:
```
pip install -U sentence-transformers
```
Then you can use the model like this:
```python
from sentence_transformers import CrossEncoder
model = CrossEncoder('itdainb/vietnamese-cross-encoder', max_length=256)
scores = model.predict([('Query', 'Paragraph1'), ('Query', 'Paragraph2') , ('Query', 'Paragraph3')])
``` |