Update README.md
Browse files
README.md
CHANGED
@@ -34,9 +34,10 @@ datasets:
|
|
34 |
The model should be selected considering search relevance, model inference and retrieval efficiency(FLOPS). We benchmark models' performance on **MIRACL** benchmark (we exclude th since the uncased backbone can not encode it).
|
35 |
**We recommend to use it with max_ratio pruning.**
|
36 |
|
37 |
-
| Model | Inference-free for Retrieval | Model Parameters | AVG NDCG@10 | AVG FLOPS |
|
38 |
-
|-------|------------------------------|------------------|-------------|-----------|
|
39 |
-
| [opensearch-neural-sparse-encoding-multilingual-v1](https://huggingface.co/opensearch-project/opensearch-neural-sparse-encoding-multilingual-v1) | ✔️ | 160M | 0.629 | 1.3 |
|
|
|
40 |
|
41 |
## Overview
|
42 |
|
@@ -156,7 +157,7 @@ The above code sample shows an example of neural sparse search. Although there i
|
|
156 |
|-------|---------|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|
|
157 |
| BM25 | 0.305 | 0.482 | 0.383 | 0.077 | 0.115 | 0.297 | 0.350 | 0.256 | 0.395 | 0.175 | 0.287 | 0.312 | 0.458 | 0.351 | 0.371 | 0.267 |
|
158 |
| [opensearch-neural-sparse-encoding-multilingual-v1](https://huggingface.co/opensearch-project/opensearch-neural-sparse-encoding-multilingual-v1) | 0.629 | 0.670 | 0.740 | 0.542 | 0.558 | 0.582 | 0.486 | 0.658 | 0.740 | 0.562 | 0.514 | 0.669 | 0.767 | 0.768 | 0.607 | 0.575 |
|
159 |
-
| [opensearch-neural-sparse-encoding-multilingual-v1](https://huggingface.co/opensearch-project/opensearch-neural-sparse-encoding-multilingual-v1); prune_ratio 0.1 | 0.
|
160 |
|
161 |
</div>
|
162 |
|
|
|
34 |
The model should be selected considering search relevance, model inference and retrieval efficiency(FLOPS). We benchmark models' performance on **MIRACL** benchmark (we exclude th since the uncased backbone can not encode it).
|
35 |
**We recommend to use it with max_ratio pruning.**
|
36 |
|
37 |
+
| Model | Inference-free for Retrieval | Model Parameters | AVG NDCG@10 | AVG FLOPS | AVG EMB SIZE |
|
38 |
+
|-------|------------------------------|------------------|-------------|-----------| -----------|
|
39 |
+
| [opensearch-neural-sparse-encoding-multilingual-v1](https://huggingface.co/opensearch-project/opensearch-neural-sparse-encoding-multilingual-v1) | ✔️ | 160M | 0.629 | 1.3 | 138 |
|
40 |
+
| [opensearch-neural-sparse-encoding-multilingual-v1](https://huggingface.co/opensearch-project/opensearch-neural-sparse-encoding-multilingual-v1); prune_ratio 0.1 | ✔️ | 160M | 0.626 | 0.8 | 75 |
|
41 |
|
42 |
## Overview
|
43 |
|
|
|
157 |
|-------|---------|----|----|----|----|----|----|----|----|----|----|----|----|----|----|----|
|
158 |
| BM25 | 0.305 | 0.482 | 0.383 | 0.077 | 0.115 | 0.297 | 0.350 | 0.256 | 0.395 | 0.175 | 0.287 | 0.312 | 0.458 | 0.351 | 0.371 | 0.267 |
|
159 |
| [opensearch-neural-sparse-encoding-multilingual-v1](https://huggingface.co/opensearch-project/opensearch-neural-sparse-encoding-multilingual-v1) | 0.629 | 0.670 | 0.740 | 0.542 | 0.558 | 0.582 | 0.486 | 0.658 | 0.740 | 0.562 | 0.514 | 0.669 | 0.767 | 0.768 | 0.607 | 0.575 |
|
160 |
+
| [opensearch-neural-sparse-encoding-multilingual-v1](https://huggingface.co/opensearch-project/opensearch-neural-sparse-encoding-multilingual-v1); prune_ratio 0.1 | 0.626 | 0.667 | 0.740 | 0.537 | 0.555 | 0.576 | 0.481 | 0.655 | 0.737 | 0.558 | 0.511 | 0.664 | 0.761 | 0.766 | 0.604 | 0.572 |
|
161 |
|
162 |
</div>
|
163 |
|