---
language:
- multilingual
- af
- am
- ar
- as
- az
- be
- bg
- bn
- br
- bs
- ca
- cs
- cy
- da
- de
- el
- en
- eo
- es
- et
- eu
- fa
- fi
- fr
- fy
- ga
- gd
- gl
- gu
- ha
- he
- hi
- hr
- hu
- hy
- id
- is
- it
- ja
- jv
- ka
- kk
- km
- kn
- ko
- ku
- ky
- la
- lo
- lt
- lv
- mg
- mk
- ml
- mn
- mr
- ms
- my
- ne
- nl
- 'no'
- om
- or
- pa
- pl
- ps
- pt
- ro
- ru
- sa
- sd
- si
- sk
- sl
- so
- sq
- sr
- su
- sv
- sw
- ta
- te
- th
- tl
- tr
- ug
- uk
- ur
- uz
- vi
- xh
- yi
- zh
license: mit
model-index:
- name: intfloat/multilingual-e5-small
results:
- dataset:
config: en
name: MTEB AmazonCounterfactualClassification (en)
revision: e8379541af4e31359cca9fbcf4b00f2671dba205
split: test
type: mteb/amazon_counterfactual
metrics:
- type: accuracy
value: 73.79104477611939
- type: ap
value: 36.9996434842022
- type: f1
value: 67.95453679103099
task:
type: Classification
- dataset:
config: de
name: MTEB AmazonCounterfactualClassification (de)
revision: e8379541af4e31359cca9fbcf4b00f2671dba205
split: test
type: mteb/amazon_counterfactual
metrics:
- type: accuracy
value: 71.64882226980728
- type: ap
value: 82.11942130026586
- type: f1
value: 69.87963421606715
task:
type: Classification
- dataset:
config: en-ext
name: MTEB AmazonCounterfactualClassification (en-ext)
revision: e8379541af4e31359cca9fbcf4b00f2671dba205
split: test
type: mteb/amazon_counterfactual
metrics:
- type: accuracy
value: 75.8095952023988
- type: ap
value: 24.46869495579561
- type: f1
value: 63.00108480037597
task:
type: Classification
- dataset:
config: ja
name: MTEB AmazonCounterfactualClassification (ja)
revision: e8379541af4e31359cca9fbcf4b00f2671dba205
split: test
type: mteb/amazon_counterfactual
metrics:
- type: accuracy
value: 64.186295503212
- type: ap
value: 15.496804690197042
- type: f1
value: 52.07153895475031
task:
type: Classification
- dataset:
config: default
name: MTEB AmazonPolarityClassification
revision: e2d317d38cd51312af73b3d32a06d1a08b442046
split: test
type: mteb/amazon_polarity
metrics:
- type: accuracy
value: 88.699325
- type: ap
value: 85.27039559917269
- type: f1
value: 88.65556295032513
task:
type: Classification
- dataset:
config: en
name: MTEB AmazonReviewsClassification (en)
revision: 1399c76144fd37290681b995c656ef9b2e06e26d
split: test
type: mteb/amazon_reviews_multi
metrics:
- type: accuracy
value: 44.69799999999999
- type: f1
value: 43.73187348654165
task:
type: Classification
- dataset:
config: de
name: MTEB AmazonReviewsClassification (de)
revision: 1399c76144fd37290681b995c656ef9b2e06e26d
split: test
type: mteb/amazon_reviews_multi
metrics:
- type: accuracy
value: 40.245999999999995
- type: f1
value: 39.3863530637684
task:
type: Classification
- dataset:
config: es
name: MTEB AmazonReviewsClassification (es)
revision: 1399c76144fd37290681b995c656ef9b2e06e26d
split: test
type: mteb/amazon_reviews_multi
metrics:
- type: accuracy
value: 40.394
- type: f1
value: 39.301223469483446
task:
type: Classification
- dataset:
config: fr
name: MTEB AmazonReviewsClassification (fr)
revision: 1399c76144fd37290681b995c656ef9b2e06e26d
split: test
type: mteb/amazon_reviews_multi
metrics:
- type: accuracy
value: 38.864
- type: f1
value: 37.97974261868003
task:
type: Classification
- dataset:
config: ja
name: MTEB AmazonReviewsClassification (ja)
revision: 1399c76144fd37290681b995c656ef9b2e06e26d
split: test
type: mteb/amazon_reviews_multi
metrics:
- type: accuracy
value: 37.682
- type: f1
value: 37.07399369768313
task:
type: Classification
- dataset:
config: zh
name: MTEB AmazonReviewsClassification (zh)
revision: 1399c76144fd37290681b995c656ef9b2e06e26d
split: test
type: mteb/amazon_reviews_multi
metrics:
- type: accuracy
value: 37.504
- type: f1
value: 36.62317273874278
task:
type: Classification
- dataset:
config: default
name: MTEB ArguAna
revision: None
split: test
type: arguana
metrics:
- type: map_at_1
value: 19.061
- type: map_at_10
value: 31.703
- type: map_at_100
value: 32.967
- type: map_at_1000
value: 33.001000000000005
- type: map_at_3
value: 27.466
- type: map_at_5
value: 29.564
- type: mrr_at_1
value: 19.559
- type: mrr_at_10
value: 31.874999999999996
- type: mrr_at_100
value: 33.146
- type: mrr_at_1000
value: 33.18
- type: mrr_at_3
value: 27.667
- type: mrr_at_5
value: 29.74
- type: ndcg_at_1
value: 19.061
- type: ndcg_at_10
value: 39.062999999999995
- type: ndcg_at_100
value: 45.184000000000005
- type: ndcg_at_1000
value: 46.115
- type: ndcg_at_3
value: 30.203000000000003
- type: ndcg_at_5
value: 33.953
- type: precision_at_1
value: 19.061
- type: precision_at_10
value: 6.279999999999999
- type: precision_at_100
value: 0.9129999999999999
- type: precision_at_1000
value: 0.099
- type: precision_at_3
value: 12.706999999999999
- type: precision_at_5
value: 9.431000000000001
- type: recall_at_1
value: 19.061
- type: recall_at_10
value: 62.802
- type: recall_at_100
value: 91.323
- type: recall_at_1000
value: 98.72
- type: recall_at_3
value: 38.122
- type: recall_at_5
value: 47.155
task:
type: Retrieval
- dataset:
config: default
name: MTEB ArxivClusteringP2P
revision: a122ad7f3f0291bf49cc6f4d32aa80929df69d5d
split: test
type: mteb/arxiv-clustering-p2p
metrics:
- type: v_measure
value: 39.22266660528253
task:
type: Clustering
- dataset:
config: default
name: MTEB ArxivClusteringS2S
revision: f910caf1a6075f7329cdf8c1a6135696f37dbd53
split: test
type: mteb/arxiv-clustering-s2s
metrics:
- type: v_measure
value: 30.79980849482483
task:
type: Clustering
- dataset:
config: default
name: MTEB AskUbuntuDupQuestions
revision: 2000358ca161889fa9c082cb41daa8dcfb161a54
split: test
type: mteb/askubuntudupquestions-reranking
metrics:
- type: map
value: 57.8790068352054
- type: mrr
value: 71.78791276436706
task:
type: Reranking
- dataset:
config: default
name: MTEB BIOSSES
revision: d3fb88f8f02e40887cd149695127462bbcf29b4a
split: test
type: mteb/biosses-sts
metrics:
- type: cos_sim_pearson
value: 82.36328364043163
- type: cos_sim_spearman
value: 82.26211536195868
- type: euclidean_pearson
value: 80.3183865039173
- type: euclidean_spearman
value: 79.88495276296132
- type: manhattan_pearson
value: 80.14484480692127
- type: manhattan_spearman
value: 80.39279565980743
task:
type: STS
- dataset:
config: de-en
name: MTEB BUCC (de-en)
revision: d51519689f32196a32af33b075a01d0e7c51e252
split: test
type: mteb/bucc-bitext-mining
metrics:
- type: accuracy
value: 98.0375782881002
- type: f1
value: 97.86012526096033
- type: precision
value: 97.77139874739039
- type: recall
value: 98.0375782881002
task:
type: BitextMining
- dataset:
config: fr-en
name: MTEB BUCC (fr-en)
revision: d51519689f32196a32af33b075a01d0e7c51e252
split: test
type: mteb/bucc-bitext-mining
metrics:
- type: accuracy
value: 93.35241030156286
- type: f1
value: 92.66050333846944
- type: precision
value: 92.3306919069631
- type: recall
value: 93.35241030156286
task:
type: BitextMining
- dataset:
config: ru-en
name: MTEB BUCC (ru-en)
revision: d51519689f32196a32af33b075a01d0e7c51e252
split: test
type: mteb/bucc-bitext-mining
metrics:
- type: accuracy
value: 94.0699688257707
- type: f1
value: 93.50236693222492
- type: precision
value: 93.22791825424315
- type: recall
value: 94.0699688257707
task:
type: BitextMining
- dataset:
config: zh-en
name: MTEB BUCC (zh-en)
revision: d51519689f32196a32af33b075a01d0e7c51e252
split: test
type: mteb/bucc-bitext-mining
metrics:
- type: accuracy
value: 89.25750394944708
- type: f1
value: 88.79234684921889
- type: precision
value: 88.57293312269616
- type: recall
value: 89.25750394944708
task:
type: BitextMining
- dataset:
config: default
name: MTEB Banking77Classification
revision: 0fd18e25b25c072e09e0d92ab615fda904d66300
split: test
type: mteb/banking77
metrics:
- type: accuracy
value: 79.41558441558442
- type: f1
value: 79.25886487487219
task:
type: Classification
- dataset:
config: default
name: MTEB BiorxivClusteringP2P
revision: 65b79d1d13f80053f67aca9498d9402c2d9f1f40
split: test
type: mteb/biorxiv-clustering-p2p
metrics:
- type: v_measure
value: 35.747820820329736
task:
type: Clustering
- dataset:
config: default
name: MTEB BiorxivClusteringS2S
revision: 258694dd0231531bc1fd9de6ceb52a0853c6d908
split: test
type: mteb/biorxiv-clustering-s2s
metrics:
- type: v_measure
value: 27.045143830596146
task:
type: Clustering
- dataset:
config: default
name: MTEB CQADupstackRetrieval
revision: None
split: test
type: BeIR/cqadupstack
metrics:
- type: map_at_1
value: 24.252999999999997
- type: map_at_10
value: 31.655916666666666
- type: map_at_100
value: 32.680749999999996
- type: map_at_1000
value: 32.79483333333334
- type: map_at_3
value: 29.43691666666666
- type: map_at_5
value: 30.717416666666665
- type: mrr_at_1
value: 28.602750000000004
- type: mrr_at_10
value: 35.56875
- type: mrr_at_100
value: 36.3595
- type: mrr_at_1000
value: 36.427749999999996
- type: mrr_at_3
value: 33.586166666666664
- type: mrr_at_5
value: 34.73641666666666
- type: ndcg_at_1
value: 28.602750000000004
- type: ndcg_at_10
value: 36.06933333333334
- type: ndcg_at_100
value: 40.70141666666667
- type: ndcg_at_1000
value: 43.24341666666667
- type: ndcg_at_3
value: 32.307916666666664
- type: ndcg_at_5
value: 34.129999999999995
- type: precision_at_1
value: 28.602750000000004
- type: precision_at_10
value: 6.097666666666667
- type: precision_at_100
value: 0.9809166666666668
- type: precision_at_1000
value: 0.13766666666666663
- type: precision_at_3
value: 14.628166666666667
- type: precision_at_5
value: 10.266916666666667
- type: recall_at_1
value: 24.252999999999997
- type: recall_at_10
value: 45.31916666666667
- type: recall_at_100
value: 66.03575000000001
- type: recall_at_1000
value: 83.94708333333334
- type: recall_at_3
value: 34.71941666666666
- type: recall_at_5
value: 39.46358333333333
task:
type: Retrieval
- dataset:
config: default
name: MTEB ClimateFEVER
revision: None
split: test
type: climate-fever
metrics:
- type: map_at_1
value: 9.024000000000001
- type: map_at_10
value: 15.644
- type: map_at_100
value: 17.154
- type: map_at_1000
value: 17.345
- type: map_at_3
value: 13.028
- type: map_at_5
value: 14.251
- type: mrr_at_1
value: 19.674
- type: mrr_at_10
value: 29.826999999999998
- type: mrr_at_100
value: 30.935000000000002
- type: mrr_at_1000
value: 30.987
- type: mrr_at_3
value: 26.645000000000003
- type: mrr_at_5
value: 28.29
- type: ndcg_at_1
value: 19.674
- type: ndcg_at_10
value: 22.545
- type: ndcg_at_100
value: 29.207
- type: ndcg_at_1000
value: 32.912
- type: ndcg_at_3
value: 17.952
- type: ndcg_at_5
value: 19.363
- type: precision_at_1
value: 19.674
- type: precision_at_10
value: 7.212000000000001
- type: precision_at_100
value: 1.435
- type: precision_at_1000
value: 0.212
- type: precision_at_3
value: 13.507
- type: precision_at_5
value: 10.397
- type: recall_at_1
value: 9.024000000000001
- type: recall_at_10
value: 28.077999999999996
- type: recall_at_100
value: 51.403
- type: recall_at_1000
value: 72.406
- type: recall_at_3
value: 16.768
- type: recall_at_5
value: 20.737
task:
type: Retrieval
- dataset:
config: default
name: MTEB DBPedia
revision: None
split: test
type: dbpedia-entity
metrics:
- type: map_at_1
value: 8.012
- type: map_at_10
value: 17.138
- type: map_at_100
value: 24.146
- type: map_at_1000
value: 25.622
- type: map_at_3
value: 12.552
- type: map_at_5
value: 14.435
- type: mrr_at_1
value: 62.25000000000001
- type: mrr_at_10
value: 71.186
- type: mrr_at_100
value: 71.504
- type: mrr_at_1000
value: 71.514
- type: mrr_at_3
value: 69.333
- type: mrr_at_5
value: 70.408
- type: ndcg_at_1
value: 49.75
- type: ndcg_at_10
value: 37.76
- type: ndcg_at_100
value: 42.071
- type: ndcg_at_1000
value: 49.309
- type: ndcg_at_3
value: 41.644
- type: ndcg_at_5
value: 39.812999999999995
- type: precision_at_1
value: 62.25000000000001
- type: precision_at_10
value: 30.15
- type: precision_at_100
value: 9.753
- type: precision_at_1000
value: 1.9189999999999998
- type: precision_at_3
value: 45.667
- type: precision_at_5
value: 39.15
- type: recall_at_1
value: 8.012
- type: recall_at_10
value: 22.599
- type: recall_at_100
value: 48.068
- type: recall_at_1000
value: 71.328
- type: recall_at_3
value: 14.043
- type: recall_at_5
value: 17.124
task:
type: Retrieval
- dataset:
config: default
name: MTEB EmotionClassification
revision: 4f58c6b202a23cf9a4da393831edf4f9183cad37
split: test
type: mteb/emotion
metrics:
- type: accuracy
value: 42.455
- type: f1
value: 37.59462649781862
task:
type: Classification
- dataset:
config: default
name: MTEB FEVER
revision: None
split: test
type: fever
metrics:
- type: map_at_1
value: 58.092
- type: map_at_10
value: 69.586
- type: map_at_100
value: 69.968
- type: map_at_1000
value: 69.982
- type: map_at_3
value: 67.48100000000001
- type: map_at_5
value: 68.915
- type: mrr_at_1
value: 62.166
- type: mrr_at_10
value: 73.588
- type: mrr_at_100
value: 73.86399999999999
- type: mrr_at_1000
value: 73.868
- type: mrr_at_3
value: 71.6
- type: mrr_at_5
value: 72.99
- type: ndcg_at_1
value: 62.166
- type: ndcg_at_10
value: 75.27199999999999
- type: ndcg_at_100
value: 76.816
- type: ndcg_at_1000
value: 77.09700000000001
- type: ndcg_at_3
value: 71.36
- type: ndcg_at_5
value: 73.785
- type: precision_at_1
value: 62.166
- type: precision_at_10
value: 9.716
- type: precision_at_100
value: 1.065
- type: precision_at_1000
value: 0.11
- type: precision_at_3
value: 28.278
- type: precision_at_5
value: 18.343999999999998
- type: recall_at_1
value: 58.092
- type: recall_at_10
value: 88.73400000000001
- type: recall_at_100
value: 95.195
- type: recall_at_1000
value: 97.04599999999999
- type: recall_at_3
value: 78.45
- type: recall_at_5
value: 84.316
task:
type: Retrieval
- dataset:
config: default
name: MTEB FiQA2018
revision: None
split: test
type: fiqa
metrics:
- type: map_at_1
value: 16.649
- type: map_at_10
value: 26.457000000000004
- type: map_at_100
value: 28.169
- type: map_at_1000
value: 28.352
- type: map_at_3
value: 23.305
- type: map_at_5
value: 25.169000000000004
- type: mrr_at_1
value: 32.407000000000004
- type: mrr_at_10
value: 40.922
- type: mrr_at_100
value: 41.931000000000004
- type: mrr_at_1000
value: 41.983
- type: mrr_at_3
value: 38.786
- type: mrr_at_5
value: 40.205999999999996
- type: ndcg_at_1
value: 32.407000000000004
- type: ndcg_at_10
value: 33.314
- type: ndcg_at_100
value: 40.312
- type: ndcg_at_1000
value: 43.685
- type: ndcg_at_3
value: 30.391000000000002
- type: ndcg_at_5
value: 31.525
- type: precision_at_1
value: 32.407000000000004
- type: precision_at_10
value: 8.966000000000001
- type: precision_at_100
value: 1.6019999999999999
- type: precision_at_1000
value: 0.22200000000000003
- type: precision_at_3
value: 20.165
- type: precision_at_5
value: 14.722
- type: recall_at_1
value: 16.649
- type: recall_at_10
value: 39.117000000000004
- type: recall_at_100
value: 65.726
- type: recall_at_1000
value: 85.784
- type: recall_at_3
value: 27.914
- type: recall_at_5
value: 33.289
task:
type: Retrieval
- dataset:
config: default
name: MTEB HotpotQA
revision: None
split: test
type: hotpotqa
metrics:
- type: map_at_1
value: 36.253
- type: map_at_10
value: 56.16799999999999
- type: map_at_100
value: 57.06099999999999
- type: map_at_1000
value: 57.126
- type: map_at_3
value: 52.644999999999996
- type: map_at_5
value: 54.909
- type: mrr_at_1
value: 72.505
- type: mrr_at_10
value: 79.66
- type: mrr_at_100
value: 79.869
- type: mrr_at_1000
value: 79.88
- type: mrr_at_3
value: 78.411
- type: mrr_at_5
value: 79.19800000000001
- type: ndcg_at_1
value: 72.505
- type: ndcg_at_10
value: 65.094
- type: ndcg_at_100
value: 68.219
- type: ndcg_at_1000
value: 69.515
- type: ndcg_at_3
value: 59.99
- type: ndcg_at_5
value: 62.909000000000006
- type: precision_at_1
value: 72.505
- type: precision_at_10
value: 13.749
- type: precision_at_100
value: 1.619
- type: precision_at_1000
value: 0.179
- type: precision_at_3
value: 38.357
- type: precision_at_5
value: 25.313000000000002
- type: recall_at_1
value: 36.253
- type: recall_at_10
value: 68.744
- type: recall_at_100
value: 80.925
- type: recall_at_1000
value: 89.534
- type: recall_at_3
value: 57.535000000000004
- type: recall_at_5
value: 63.282000000000004
task:
type: Retrieval
- dataset:
config: default
name: MTEB ImdbClassification
revision: 3d86128a09e091d6018b6d26cad27f2739fc2db7
split: test
type: mteb/imdb
metrics:
- type: accuracy
value: 80.82239999999999
- type: ap
value: 75.65895781725314
- type: f1
value: 80.75880969095746
task:
type: Classification
- dataset:
config: default
name: MTEB MSMARCO
revision: None
split: dev
type: msmarco
metrics:
- type: map_at_1
value: 21.624
- type: map_at_10
value: 34.075
- type: map_at_100
value: 35.229
- type: map_at_1000
value: 35.276999999999994
- type: map_at_3
value: 30.245
- type: map_at_5
value: 32.42
- type: mrr_at_1
value: 22.264
- type: mrr_at_10
value: 34.638000000000005
- type: mrr_at_100
value: 35.744
- type: mrr_at_1000
value: 35.787
- type: mrr_at_3
value: 30.891000000000002
- type: mrr_at_5
value: 33.042
- type: ndcg_at_1
value: 22.264
- type: ndcg_at_10
value: 40.991
- type: ndcg_at_100
value: 46.563
- type: ndcg_at_1000
value: 47.743
- type: ndcg_at_3
value: 33.198
- type: ndcg_at_5
value: 37.069
- type: precision_at_1
value: 22.264
- type: precision_at_10
value: 6.5089999999999995
- type: precision_at_100
value: 0.9299999999999999
- type: precision_at_1000
value: 0.10300000000000001
- type: precision_at_3
value: 14.216999999999999
- type: precision_at_5
value: 10.487
- type: recall_at_1
value: 21.624
- type: recall_at_10
value: 62.303
- type: recall_at_100
value: 88.124
- type: recall_at_1000
value: 97.08
- type: recall_at_3
value: 41.099999999999994
- type: recall_at_5
value: 50.381
task:
type: Retrieval
- dataset:
config: en
name: MTEB MTOPDomainClassification (en)
revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
split: test
type: mteb/mtop_domain
metrics:
- type: accuracy
value: 91.06703146374831
- type: f1
value: 90.86867815863172
task:
type: Classification
- dataset:
config: de
name: MTEB MTOPDomainClassification (de)
revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
split: test
type: mteb/mtop_domain
metrics:
- type: accuracy
value: 87.46970977740209
- type: f1
value: 86.36832872036588
task:
type: Classification
- dataset:
config: es
name: MTEB MTOPDomainClassification (es)
revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
split: test
type: mteb/mtop_domain
metrics:
- type: accuracy
value: 89.26951300867245
- type: f1
value: 88.93561193959502
task:
type: Classification
- dataset:
config: fr
name: MTEB MTOPDomainClassification (fr)
revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
split: test
type: mteb/mtop_domain
metrics:
- type: accuracy
value: 84.22799874725963
- type: f1
value: 84.30490069236556
task:
type: Classification
- dataset:
config: hi
name: MTEB MTOPDomainClassification (hi)
revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
split: test
type: mteb/mtop_domain
metrics:
- type: accuracy
value: 86.02007888131948
- type: f1
value: 85.39376041027991
task:
type: Classification
- dataset:
config: th
name: MTEB MTOPDomainClassification (th)
revision: d80d48c1eb48d3562165c59d59d0034df9fff0bf
split: test
type: mteb/mtop_domain
metrics:
- type: accuracy
value: 85.34900542495481
- type: f1
value: 85.39859673336713
task:
type: Classification
- dataset:
config: en
name: MTEB MTOPIntentClassification (en)
revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
split: test
type: mteb/mtop_intent
metrics:
- type: accuracy
value: 71.078431372549
- type: f1
value: 53.45071102002276
task:
type: Classification
- dataset:
config: de
name: MTEB MTOPIntentClassification (de)
revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
split: test
type: mteb/mtop_intent
metrics:
- type: accuracy
value: 65.85798816568047
- type: f1
value: 46.53112748993529
task:
type: Classification
- dataset:
config: es
name: MTEB MTOPIntentClassification (es)
revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
split: test
type: mteb/mtop_intent
metrics:
- type: accuracy
value: 67.96864576384256
- type: f1
value: 45.966703022829506
task:
type: Classification
- dataset:
config: fr
name: MTEB MTOPIntentClassification (fr)
revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
split: test
type: mteb/mtop_intent
metrics:
- type: accuracy
value: 61.31537738803633
- type: f1
value: 45.52601712835461
task:
type: Classification
- dataset:
config: hi
name: MTEB MTOPIntentClassification (hi)
revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
split: test
type: mteb/mtop_intent
metrics:
- type: accuracy
value: 66.29616349946218
- type: f1
value: 47.24166485726613
task:
type: Classification
- dataset:
config: th
name: MTEB MTOPIntentClassification (th)
revision: ae001d0e6b1228650b7bd1c2c65fb50ad11a8aba
split: test
type: mteb/mtop_intent
metrics:
- type: accuracy
value: 67.51537070524412
- type: f1
value: 49.463476319014276
task:
type: Classification
- dataset:
config: af
name: MTEB MassiveIntentClassification (af)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 57.06792199058508
- type: f1
value: 54.094921857502285
task:
type: Classification
- dataset:
config: am
name: MTEB MassiveIntentClassification (am)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 51.960322797579025
- type: f1
value: 48.547371223370945
task:
type: Classification
- dataset:
config: ar
name: MTEB MassiveIntentClassification (ar)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 54.425016812373904
- type: f1
value: 50.47069202054312
task:
type: Classification
- dataset:
config: az
name: MTEB MassiveIntentClassification (az)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 59.798251513113655
- type: f1
value: 57.05013069086648
task:
type: Classification
- dataset:
config: bn
name: MTEB MassiveIntentClassification (bn)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 59.37794216543376
- type: f1
value: 56.3607992649805
task:
type: Classification
- dataset:
config: cy
name: MTEB MassiveIntentClassification (cy)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 46.56018829858777
- type: f1
value: 43.87319715715134
task:
type: Classification
- dataset:
config: da
name: MTEB MassiveIntentClassification (da)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 62.9724277067922
- type: f1
value: 59.36480066245562
task:
type: Classification
- dataset:
config: de
name: MTEB MassiveIntentClassification (de)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 62.72696704774715
- type: f1
value: 59.143595966615855
task:
type: Classification
- dataset:
config: el
name: MTEB MassiveIntentClassification (el)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 61.5971755211836
- type: f1
value: 59.169445724946726
task:
type: Classification
- dataset:
config: en
name: MTEB MassiveIntentClassification (en)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 70.29589778076665
- type: f1
value: 67.7577001808977
task:
type: Classification
- dataset:
config: es
name: MTEB MassiveIntentClassification (es)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 66.31136516476126
- type: f1
value: 64.52032955983242
task:
type: Classification
- dataset:
config: fa
name: MTEB MassiveIntentClassification (fa)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 65.54472091459314
- type: f1
value: 61.47903120066317
task:
type: Classification
- dataset:
config: fi
name: MTEB MassiveIntentClassification (fi)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 61.45595158036314
- type: f1
value: 58.0891846024637
task:
type: Classification
- dataset:
config: fr
name: MTEB MassiveIntentClassification (fr)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 65.47074646940149
- type: f1
value: 62.84830858877575
task:
type: Classification
- dataset:
config: he
name: MTEB MassiveIntentClassification (he)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 58.046402151983855
- type: f1
value: 55.269074430533195
task:
type: Classification
- dataset:
config: hi
name: MTEB MassiveIntentClassification (hi)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 64.06523201075991
- type: f1
value: 61.35339643021369
task:
type: Classification
- dataset:
config: hu
name: MTEB MassiveIntentClassification (hu)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 60.954942837928726
- type: f1
value: 57.07035922704846
task:
type: Classification
- dataset:
config: hy
name: MTEB MassiveIntentClassification (hy)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 57.404169468728995
- type: f1
value: 53.94259011839138
task:
type: Classification
- dataset:
config: id
name: MTEB MassiveIntentClassification (id)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 64.16610625420309
- type: f1
value: 61.337103431499365
task:
type: Classification
- dataset:
config: is
name: MTEB MassiveIntentClassification (is)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 52.262945527908535
- type: f1
value: 49.7610691598921
task:
type: Classification
- dataset:
config: it
name: MTEB MassiveIntentClassification (it)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 65.54472091459314
- type: f1
value: 63.469099018440154
task:
type: Classification
- dataset:
config: ja
name: MTEB MassiveIntentClassification (ja)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 68.22797579018157
- type: f1
value: 64.89098471083001
task:
type: Classification
- dataset:
config: jv
name: MTEB MassiveIntentClassification (jv)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 50.847343644922674
- type: f1
value: 47.8536963168393
task:
type: Classification
- dataset:
config: ka
name: MTEB MassiveIntentClassification (ka)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 48.45326160053799
- type: f1
value: 46.370078045805556
task:
type: Classification
- dataset:
config: km
name: MTEB MassiveIntentClassification (km)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 42.83120376597175
- type: f1
value: 39.68948521599982
task:
type: Classification
- dataset:
config: kn
name: MTEB MassiveIntentClassification (kn)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 57.5084061869536
- type: f1
value: 53.961876160401545
task:
type: Classification
- dataset:
config: ko
name: MTEB MassiveIntentClassification (ko)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 63.7895090786819
- type: f1
value: 61.134223684676
task:
type: Classification
- dataset:
config: lv
name: MTEB MassiveIntentClassification (lv)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 54.98991257565569
- type: f1
value: 52.579862862826296
task:
type: Classification
- dataset:
config: ml
name: MTEB MassiveIntentClassification (ml)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 61.90316072629456
- type: f1
value: 58.203024538290336
task:
type: Classification
- dataset:
config: mn
name: MTEB MassiveIntentClassification (mn)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 57.09818426361802
- type: f1
value: 54.22718458445455
task:
type: Classification
- dataset:
config: ms
name: MTEB MassiveIntentClassification (ms)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 58.991257565568255
- type: f1
value: 55.84892781767421
task:
type: Classification
- dataset:
config: my
name: MTEB MassiveIntentClassification (my)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 55.901143241425686
- type: f1
value: 52.25264332199797
task:
type: Classification
- dataset:
config: nb
name: MTEB MassiveIntentClassification (nb)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 61.96368527236047
- type: f1
value: 58.927243876153454
task:
type: Classification
- dataset:
config: nl
name: MTEB MassiveIntentClassification (nl)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 65.64223268325489
- type: f1
value: 62.340453718379706
task:
type: Classification
- dataset:
config: pl
name: MTEB MassiveIntentClassification (pl)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 64.52589105581708
- type: f1
value: 61.661113187022174
task:
type: Classification
- dataset:
config: pt
name: MTEB MassiveIntentClassification (pt)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 66.84599865501009
- type: f1
value: 64.59342572873005
task:
type: Classification
- dataset:
config: ro
name: MTEB MassiveIntentClassification (ro)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 60.81035642232684
- type: f1
value: 57.5169089806797
task:
type: Classification
- dataset:
config: ru
name: MTEB MassiveIntentClassification (ru)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 58.652238071815056
- type: f1
value: 53.22732406426353
- type: f1_weighted
value: 57.585586737209546
- type: main_score
value: 58.652238071815056
task:
type: Classification
- dataset:
config: sl
name: MTEB MassiveIntentClassification (sl)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 56.51647612642906
- type: f1
value: 54.33154780100043
task:
type: Classification
- dataset:
config: sq
name: MTEB MassiveIntentClassification (sq)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 57.985877605917956
- type: f1
value: 54.46187524463802
task:
type: Classification
- dataset:
config: sv
name: MTEB MassiveIntentClassification (sv)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 65.03026227303296
- type: f1
value: 62.34377392877748
task:
type: Classification
- dataset:
config: sw
name: MTEB MassiveIntentClassification (sw)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 53.567585743106925
- type: f1
value: 50.73770655983206
task:
type: Classification
- dataset:
config: ta
name: MTEB MassiveIntentClassification (ta)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 57.2595830531271
- type: f1
value: 53.657327291708626
task:
type: Classification
- dataset:
config: te
name: MTEB MassiveIntentClassification (te)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 57.82784129119032
- type: f1
value: 54.82518072665301
task:
type: Classification
- dataset:
config: th
name: MTEB MassiveIntentClassification (th)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 64.06859448554137
- type: f1
value: 63.00185280500495
task:
type: Classification
- dataset:
config: tl
name: MTEB MassiveIntentClassification (tl)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 58.91055817081371
- type: f1
value: 55.54116301224262
task:
type: Classification
- dataset:
config: tr
name: MTEB MassiveIntentClassification (tr)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 63.54404841963686
- type: f1
value: 59.57650946030184
task:
type: Classification
- dataset:
config: ur
name: MTEB MassiveIntentClassification (ur)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 59.27706792199059
- type: f1
value: 56.50010066083435
task:
type: Classification
- dataset:
config: vi
name: MTEB MassiveIntentClassification (vi)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 64.0719569603228
- type: f1
value: 61.817075925647956
task:
type: Classification
- dataset:
config: zh-CN
name: MTEB MassiveIntentClassification (zh-CN)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 68.23806321452591
- type: f1
value: 65.24917026029749
task:
type: Classification
- dataset:
config: zh-TW
name: MTEB MassiveIntentClassification (zh-TW)
revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
split: test
type: mteb/amazon_massive_intent
metrics:
- type: accuracy
value: 62.53530598520511
- type: f1
value: 61.71131132295768
task:
type: Classification
- dataset:
config: af
name: MTEB MassiveScenarioClassification (af)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 63.04303967720243
- type: f1
value: 60.3950085685985
task:
type: Classification
- dataset:
config: am
name: MTEB MassiveScenarioClassification (am)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 56.83591123066578
- type: f1
value: 54.95059828830849
task:
type: Classification
- dataset:
config: ar
name: MTEB MassiveScenarioClassification (ar)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 59.62340282447881
- type: f1
value: 59.525159996498225
task:
type: Classification
- dataset:
config: az
name: MTEB MassiveScenarioClassification (az)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 60.85406859448555
- type: f1
value: 59.129299095681276
task:
type: Classification
- dataset:
config: bn
name: MTEB MassiveScenarioClassification (bn)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 62.76731674512441
- type: f1
value: 61.159560612627715
task:
type: Classification
- dataset:
config: cy
name: MTEB MassiveScenarioClassification (cy)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 50.181573638197705
- type: f1
value: 46.98422176289957
task:
type: Classification
- dataset:
config: da
name: MTEB MassiveScenarioClassification (da)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 68.92737054472092
- type: f1
value: 67.69135611952979
task:
type: Classification
- dataset:
config: de
name: MTEB MassiveScenarioClassification (de)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 69.18964357767318
- type: f1
value: 68.46106138186214
task:
type: Classification
- dataset:
config: el
name: MTEB MassiveScenarioClassification (el)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 67.0712844653665
- type: f1
value: 66.75545422473901
task:
type: Classification
- dataset:
config: en
name: MTEB MassiveScenarioClassification (en)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 74.4754539340955
- type: f1
value: 74.38427146553252
task:
type: Classification
- dataset:
config: es
name: MTEB MassiveScenarioClassification (es)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 69.82515131136518
- type: f1
value: 69.63516462173847
task:
type: Classification
- dataset:
config: fa
name: MTEB MassiveScenarioClassification (fa)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 68.70880968392737
- type: f1
value: 67.45420662567926
task:
type: Classification
- dataset:
config: fi
name: MTEB MassiveScenarioClassification (fi)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 65.95494283792871
- type: f1
value: 65.06191009049222
task:
type: Classification
- dataset:
config: fr
name: MTEB MassiveScenarioClassification (fr)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 68.75924680564896
- type: f1
value: 68.30833379585945
task:
type: Classification
- dataset:
config: he
name: MTEB MassiveScenarioClassification (he)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 63.806321452589096
- type: f1
value: 63.273048243765054
task:
type: Classification
- dataset:
config: hi
name: MTEB MassiveScenarioClassification (hi)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 67.68997982515133
- type: f1
value: 66.54703855381324
task:
type: Classification
- dataset:
config: hu
name: MTEB MassiveScenarioClassification (hu)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 66.46940147948891
- type: f1
value: 65.91017343463396
task:
type: Classification
- dataset:
config: hy
name: MTEB MassiveScenarioClassification (hy)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 59.49899125756556
- type: f1
value: 57.90333469917769
task:
type: Classification
- dataset:
config: id
name: MTEB MassiveScenarioClassification (id)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 67.9219905850706
- type: f1
value: 67.23169403762938
task:
type: Classification
- dataset:
config: is
name: MTEB MassiveScenarioClassification (is)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 56.486213853396094
- type: f1
value: 54.85282355583758
task:
type: Classification
- dataset:
config: it
name: MTEB MassiveScenarioClassification (it)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 69.04169468728985
- type: f1
value: 68.83833333320462
task:
type: Classification
- dataset:
config: ja
name: MTEB MassiveScenarioClassification (ja)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 73.88702084734365
- type: f1
value: 74.04474735232299
task:
type: Classification
- dataset:
config: jv
name: MTEB MassiveScenarioClassification (jv)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 56.63416274377943
- type: f1
value: 55.11332211687954
task:
type: Classification
- dataset:
config: ka
name: MTEB MassiveScenarioClassification (ka)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 52.23604572965702
- type: f1
value: 50.86529813991055
task:
type: Classification
- dataset:
config: km
name: MTEB MassiveScenarioClassification (km)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 46.62407531943511
- type: f1
value: 43.63485467164535
task:
type: Classification
- dataset:
config: kn
name: MTEB MassiveScenarioClassification (kn)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 59.15601882985878
- type: f1
value: 57.522837510959924
task:
type: Classification
- dataset:
config: ko
name: MTEB MassiveScenarioClassification (ko)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 69.84532616005382
- type: f1
value: 69.60021127179697
task:
type: Classification
- dataset:
config: lv
name: MTEB MassiveScenarioClassification (lv)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 56.65770006724949
- type: f1
value: 55.84219135523227
task:
type: Classification
- dataset:
config: ml
name: MTEB MassiveScenarioClassification (ml)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 66.53665097511768
- type: f1
value: 65.09087787792639
task:
type: Classification
- dataset:
config: mn
name: MTEB MassiveScenarioClassification (mn)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 59.31405514458642
- type: f1
value: 58.06135303831491
task:
type: Classification
- dataset:
config: ms
name: MTEB MassiveScenarioClassification (ms)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 64.88231338264964
- type: f1
value: 62.751099407787926
task:
type: Classification
- dataset:
config: my
name: MTEB MassiveScenarioClassification (my)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 58.86012104909213
- type: f1
value: 56.29118323058282
task:
type: Classification
- dataset:
config: nb
name: MTEB MassiveScenarioClassification (nb)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 67.37390719569602
- type: f1
value: 66.27922244885102
task:
type: Classification
- dataset:
config: nl
name: MTEB MassiveScenarioClassification (nl)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 70.8675184936113
- type: f1
value: 70.22146529932019
task:
type: Classification
- dataset:
config: pl
name: MTEB MassiveScenarioClassification (pl)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 68.2212508406187
- type: f1
value: 67.77454802056282
task:
type: Classification
- dataset:
config: pt
name: MTEB MassiveScenarioClassification (pt)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 68.18090114324143
- type: f1
value: 68.03737625431621
task:
type: Classification
- dataset:
config: ro
name: MTEB MassiveScenarioClassification (ro)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 64.65030262273034
- type: f1
value: 63.792945486912856
task:
type: Classification
- dataset:
config: ru
name: MTEB MassiveScenarioClassification (ru)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 63.772749631087066
- type: f1
value: 63.4539101720024
- type: f1_weighted
value: 62.778603897469566
- type: main_score
value: 63.772749631087066
task:
type: Classification
- dataset:
config: sl
name: MTEB MassiveScenarioClassification (sl)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 60.17821116341627
- type: f1
value: 59.3935969827171
task:
type: Classification
- dataset:
config: sq
name: MTEB MassiveScenarioClassification (sq)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 62.86146603900471
- type: f1
value: 60.133692735032376
task:
type: Classification
- dataset:
config: sv
name: MTEB MassiveScenarioClassification (sv)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 70.89441829186282
- type: f1
value: 70.03064076194089
task:
type: Classification
- dataset:
config: sw
name: MTEB MassiveScenarioClassification (sw)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 58.15063887020847
- type: f1
value: 56.23326278499678
task:
type: Classification
- dataset:
config: ta
name: MTEB MassiveScenarioClassification (ta)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 59.43846671149966
- type: f1
value: 57.70440450281974
task:
type: Classification
- dataset:
config: te
name: MTEB MassiveScenarioClassification (te)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 60.8507061197041
- type: f1
value: 59.22916396061171
task:
type: Classification
- dataset:
config: th
name: MTEB MassiveScenarioClassification (th)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 70.65568258238063
- type: f1
value: 69.90736239440633
task:
type: Classification
- dataset:
config: tl
name: MTEB MassiveScenarioClassification (tl)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 60.8843308675185
- type: f1
value: 59.30332663713599
task:
type: Classification
- dataset:
config: tr
name: MTEB MassiveScenarioClassification (tr)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 68.05312710154674
- type: f1
value: 67.44024062594775
task:
type: Classification
- dataset:
config: ur
name: MTEB MassiveScenarioClassification (ur)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 62.111634162743776
- type: f1
value: 60.89083013084519
task:
type: Classification
- dataset:
config: vi
name: MTEB MassiveScenarioClassification (vi)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 67.44115669132482
- type: f1
value: 67.92227541674552
task:
type: Classification
- dataset:
config: zh-CN
name: MTEB MassiveScenarioClassification (zh-CN)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 74.4687289845326
- type: f1
value: 74.16376793486025
task:
type: Classification
- dataset:
config: zh-TW
name: MTEB MassiveScenarioClassification (zh-TW)
revision: 7d571f92784cd94a019292a1f45445077d0ef634
split: test
type: mteb/amazon_massive_scenario
metrics:
- type: accuracy
value: 68.31876260928043
- type: f1
value: 68.5246745215607
task:
type: Classification
- dataset:
config: default
name: MTEB MedrxivClusteringP2P
revision: e7a26af6f3ae46b30dde8737f02c07b1505bcc73
split: test
type: mteb/medrxiv-clustering-p2p
metrics:
- type: v_measure
value: 30.90431696479766
task:
type: Clustering
- dataset:
config: default
name: MTEB MedrxivClusteringS2S
revision: 35191c8c0dca72d8ff3efcd72aa802307d469663
split: test
type: mteb/medrxiv-clustering-s2s
metrics:
- type: v_measure
value: 27.259158476693774
task:
type: Clustering
- dataset:
config: default
name: MTEB MindSmallReranking
revision: 3bdac13927fdc888b903db93b2ffdbd90b295a69
split: test
type: mteb/mind_small
metrics:
- type: map
value: 30.28445330838555
- type: mrr
value: 31.15758529581164
task:
type: Reranking
- dataset:
config: default
name: MTEB NFCorpus
revision: None
split: test
type: nfcorpus
metrics:
- type: map_at_1
value: 5.353
- type: map_at_10
value: 11.565
- type: map_at_100
value: 14.097000000000001
- type: map_at_1000
value: 15.354999999999999
- type: map_at_3
value: 8.749
- type: map_at_5
value: 9.974
- type: mrr_at_1
value: 42.105
- type: mrr_at_10
value: 50.589
- type: mrr_at_100
value: 51.187000000000005
- type: mrr_at_1000
value: 51.233
- type: mrr_at_3
value: 48.246
- type: mrr_at_5
value: 49.546
- type: ndcg_at_1
value: 40.402
- type: ndcg_at_10
value: 31.009999999999998
- type: ndcg_at_100
value: 28.026
- type: ndcg_at_1000
value: 36.905
- type: ndcg_at_3
value: 35.983
- type: ndcg_at_5
value: 33.764
- type: precision_at_1
value: 42.105
- type: precision_at_10
value: 22.786
- type: precision_at_100
value: 6.916
- type: precision_at_1000
value: 1.981
- type: precision_at_3
value: 33.333
- type: precision_at_5
value: 28.731
- type: recall_at_1
value: 5.353
- type: recall_at_10
value: 15.039
- type: recall_at_100
value: 27.348
- type: recall_at_1000
value: 59.453
- type: recall_at_3
value: 9.792
- type: recall_at_5
value: 11.882
task:
type: Retrieval
- dataset:
config: default
name: MTEB NQ
revision: None
split: test
type: nq
metrics:
- type: map_at_1
value: 33.852
- type: map_at_10
value: 48.924
- type: map_at_100
value: 49.854
- type: map_at_1000
value: 49.886
- type: map_at_3
value: 44.9
- type: map_at_5
value: 47.387
- type: mrr_at_1
value: 38.035999999999994
- type: mrr_at_10
value: 51.644
- type: mrr_at_100
value: 52.339
- type: mrr_at_1000
value: 52.35999999999999
- type: mrr_at_3
value: 48.421
- type: mrr_at_5
value: 50.468999999999994
- type: ndcg_at_1
value: 38.007000000000005
- type: ndcg_at_10
value: 56.293000000000006
- type: ndcg_at_100
value: 60.167
- type: ndcg_at_1000
value: 60.916000000000004
- type: ndcg_at_3
value: 48.903999999999996
- type: ndcg_at_5
value: 52.978
- type: precision_at_1
value: 38.007000000000005
- type: precision_at_10
value: 9.041
- type: precision_at_100
value: 1.1199999999999999
- type: precision_at_1000
value: 0.11900000000000001
- type: precision_at_3
value: 22.084
- type: precision_at_5
value: 15.608
- type: recall_at_1
value: 33.852
- type: recall_at_10
value: 75.893
- type: recall_at_100
value: 92.589
- type: recall_at_1000
value: 98.153
- type: recall_at_3
value: 56.969
- type: recall_at_5
value: 66.283
task:
type: Retrieval
- dataset:
config: default
name: MTEB QuoraRetrieval
revision: None
split: test
type: quora
metrics:
- type: map_at_1
value: 69.174
- type: map_at_10
value: 82.891
- type: map_at_100
value: 83.545
- type: map_at_1000
value: 83.56700000000001
- type: map_at_3
value: 79.944
- type: map_at_5
value: 81.812
- type: mrr_at_1
value: 79.67999999999999
- type: mrr_at_10
value: 86.279
- type: mrr_at_100
value: 86.39
- type: mrr_at_1000
value: 86.392
- type: mrr_at_3
value: 85.21
- type: mrr_at_5
value: 85.92999999999999
- type: ndcg_at_1
value: 79.69000000000001
- type: ndcg_at_10
value: 86.929
- type: ndcg_at_100
value: 88.266
- type: ndcg_at_1000
value: 88.428
- type: ndcg_at_3
value: 83.899
- type: ndcg_at_5
value: 85.56700000000001
- type: precision_at_1
value: 79.69000000000001
- type: precision_at_10
value: 13.161000000000001
- type: precision_at_100
value: 1.513
- type: precision_at_1000
value: 0.156
- type: precision_at_3
value: 36.603
- type: precision_at_5
value: 24.138
- type: recall_at_1
value: 69.174
- type: recall_at_10
value: 94.529
- type: recall_at_100
value: 99.15
- type: recall_at_1000
value: 99.925
- type: recall_at_3
value: 85.86200000000001
- type: recall_at_5
value: 90.501
task:
type: Retrieval
- dataset:
config: default
name: MTEB RedditClustering
revision: 24640382cdbf8abc73003fb0fa6d111a705499eb
split: test
type: mteb/reddit-clustering
metrics:
- type: v_measure
value: 39.13064340585255
task:
type: Clustering
- dataset:
config: default
name: MTEB RedditClusteringP2P
revision: 282350215ef01743dc01b456c7f5241fa8937f16
split: test
type: mteb/reddit-clustering-p2p
metrics:
- type: v_measure
value: 58.97884249325877
task:
type: Clustering
- dataset:
config: default
name: MTEB SCIDOCS
revision: None
split: test
type: scidocs
metrics:
- type: map_at_1
value: 3.4680000000000004
- type: map_at_10
value: 7.865
- type: map_at_100
value: 9.332
- type: map_at_1000
value: 9.587
- type: map_at_3
value: 5.800000000000001
- type: map_at_5
value: 6.8790000000000004
- type: mrr_at_1
value: 17.0
- type: mrr_at_10
value: 25.629
- type: mrr_at_100
value: 26.806
- type: mrr_at_1000
value: 26.889000000000003
- type: mrr_at_3
value: 22.8
- type: mrr_at_5
value: 24.26
- type: ndcg_at_1
value: 17.0
- type: ndcg_at_10
value: 13.895
- type: ndcg_at_100
value: 20.491999999999997
- type: ndcg_at_1000
value: 25.759999999999998
- type: ndcg_at_3
value: 13.347999999999999
- type: ndcg_at_5
value: 11.61
- type: precision_at_1
value: 17.0
- type: precision_at_10
value: 7.090000000000001
- type: precision_at_100
value: 1.669
- type: precision_at_1000
value: 0.294
- type: precision_at_3
value: 12.3
- type: precision_at_5
value: 10.02
- type: recall_at_1
value: 3.4680000000000004
- type: recall_at_10
value: 14.363000000000001
- type: recall_at_100
value: 33.875
- type: recall_at_1000
value: 59.711999999999996
- type: recall_at_3
value: 7.483
- type: recall_at_5
value: 10.173
task:
type: Retrieval
- dataset:
config: default
name: MTEB SICK-R
revision: a6ea5a8cab320b040a23452cc28066d9beae2cee
split: test
type: mteb/sickr-sts
metrics:
- type: cos_sim_pearson
value: 83.04084311714061
- type: cos_sim_spearman
value: 77.51342467443078
- type: euclidean_pearson
value: 80.0321166028479
- type: euclidean_spearman
value: 77.29249114733226
- type: manhattan_pearson
value: 80.03105964262431
- type: manhattan_spearman
value: 77.22373689514794
task:
type: STS
- dataset:
config: default
name: MTEB STS12
revision: a0d554a64d88156834ff5ae9920b964011b16384
split: test
type: mteb/sts12-sts
metrics:
- type: cos_sim_pearson
value: 84.1680158034387
- type: cos_sim_spearman
value: 76.55983344071117
- type: euclidean_pearson
value: 79.75266678300143
- type: euclidean_spearman
value: 75.34516823467025
- type: manhattan_pearson
value: 79.75959151517357
- type: manhattan_spearman
value: 75.42330344141912
task:
type: STS
- dataset:
config: default
name: MTEB STS13
revision: 7e90230a92c190f1bf69ae9002b8cea547a64cca
split: test
type: mteb/sts13-sts
metrics:
- type: cos_sim_pearson
value: 76.48898993209346
- type: cos_sim_spearman
value: 76.96954120323366
- type: euclidean_pearson
value: 76.94139109279668
- type: euclidean_spearman
value: 76.85860283201711
- type: manhattan_pearson
value: 76.6944095091912
- type: manhattan_spearman
value: 76.61096912972553
task:
type: STS
- dataset:
config: default
name: MTEB STS14
revision: 6031580fec1f6af667f0bd2da0a551cf4f0b2375
split: test
type: mteb/sts14-sts
metrics:
- type: cos_sim_pearson
value: 77.85082366246944
- type: cos_sim_spearman
value: 75.52053350101731
- type: euclidean_pearson
value: 77.1165845070926
- type: euclidean_spearman
value: 75.31216065884388
- type: manhattan_pearson
value: 77.06193941833494
- type: manhattan_spearman
value: 75.31003701700112
task:
type: STS
- dataset:
config: default
name: MTEB STS15
revision: ae752c7c21bf194d8b67fd573edf7ae58183cbe3
split: test
type: mteb/sts15-sts
metrics:
- type: cos_sim_pearson
value: 86.36305246526497
- type: cos_sim_spearman
value: 87.11704613927415
- type: euclidean_pearson
value: 86.04199125810939
- type: euclidean_spearman
value: 86.51117572414263
- type: manhattan_pearson
value: 86.0805106816633
- type: manhattan_spearman
value: 86.52798366512229
task:
type: STS
- dataset:
config: default
name: MTEB STS16
revision: 4d8694f8f0e0100860b497b999b3dbed754a0513
split: test
type: mteb/sts16-sts
metrics:
- type: cos_sim_pearson
value: 82.18536255599724
- type: cos_sim_spearman
value: 83.63377151025418
- type: euclidean_pearson
value: 83.24657467993141
- type: euclidean_spearman
value: 84.02751481993825
- type: manhattan_pearson
value: 83.11941806582371
- type: manhattan_spearman
value: 83.84251281019304
task:
type: STS
- dataset:
config: ko-ko
name: MTEB STS17 (ko-ko)
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
split: test
type: mteb/sts17-crosslingual-sts
metrics:
- type: cos_sim_pearson
value: 78.95816528475514
- type: cos_sim_spearman
value: 78.86607380120462
- type: euclidean_pearson
value: 78.51268699230545
- type: euclidean_spearman
value: 79.11649316502229
- type: manhattan_pearson
value: 78.32367302808157
- type: manhattan_spearman
value: 78.90277699624637
task:
type: STS
- dataset:
config: ar-ar
name: MTEB STS17 (ar-ar)
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
split: test
type: mteb/sts17-crosslingual-sts
metrics:
- type: cos_sim_pearson
value: 72.89126914997624
- type: cos_sim_spearman
value: 73.0296921832678
- type: euclidean_pearson
value: 71.50385903677738
- type: euclidean_spearman
value: 73.13368899716289
- type: manhattan_pearson
value: 71.47421463379519
- type: manhattan_spearman
value: 73.03383242946575
task:
type: STS
- dataset:
config: en-ar
name: MTEB STS17 (en-ar)
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
split: test
type: mteb/sts17-crosslingual-sts
metrics:
- type: cos_sim_pearson
value: 59.22923684492637
- type: cos_sim_spearman
value: 57.41013211368396
- type: euclidean_pearson
value: 61.21107388080905
- type: euclidean_spearman
value: 60.07620768697254
- type: manhattan_pearson
value: 59.60157142786555
- type: manhattan_spearman
value: 59.14069604103739
task:
type: STS
- dataset:
config: en-de
name: MTEB STS17 (en-de)
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
split: test
type: mteb/sts17-crosslingual-sts
metrics:
- type: cos_sim_pearson
value: 76.24345978774299
- type: cos_sim_spearman
value: 77.24225743830719
- type: euclidean_pearson
value: 76.66226095469165
- type: euclidean_spearman
value: 77.60708820493146
- type: manhattan_pearson
value: 76.05303324760429
- type: manhattan_spearman
value: 76.96353149912348
task:
type: STS
- dataset:
config: en-en
name: MTEB STS17 (en-en)
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
split: test
type: mteb/sts17-crosslingual-sts
metrics:
- type: cos_sim_pearson
value: 85.50879160160852
- type: cos_sim_spearman
value: 86.43594662965224
- type: euclidean_pearson
value: 86.06846012826577
- type: euclidean_spearman
value: 86.02041395794136
- type: manhattan_pearson
value: 86.10916255616904
- type: manhattan_spearman
value: 86.07346068198953
task:
type: STS
- dataset:
config: en-tr
name: MTEB STS17 (en-tr)
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
split: test
type: mteb/sts17-crosslingual-sts
metrics:
- type: cos_sim_pearson
value: 58.39803698977196
- type: cos_sim_spearman
value: 55.96910950423142
- type: euclidean_pearson
value: 58.17941175613059
- type: euclidean_spearman
value: 55.03019330522745
- type: manhattan_pearson
value: 57.333358138183286
- type: manhattan_spearman
value: 54.04614023149965
task:
type: STS
- dataset:
config: es-en
name: MTEB STS17 (es-en)
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
split: test
type: mteb/sts17-crosslingual-sts
metrics:
- type: cos_sim_pearson
value: 70.98304089637197
- type: cos_sim_spearman
value: 72.44071656215888
- type: euclidean_pearson
value: 72.19224359033983
- type: euclidean_spearman
value: 73.89871188913025
- type: manhattan_pearson
value: 71.21098311547406
- type: manhattan_spearman
value: 72.93405764824821
task:
type: STS
- dataset:
config: es-es
name: MTEB STS17 (es-es)
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
split: test
type: mteb/sts17-crosslingual-sts
metrics:
- type: cos_sim_pearson
value: 85.99792397466308
- type: cos_sim_spearman
value: 84.83824377879495
- type: euclidean_pearson
value: 85.70043288694438
- type: euclidean_spearman
value: 84.70627558703686
- type: manhattan_pearson
value: 85.89570850150801
- type: manhattan_spearman
value: 84.95806105313007
task:
type: STS
- dataset:
config: fr-en
name: MTEB STS17 (fr-en)
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
split: test
type: mteb/sts17-crosslingual-sts
metrics:
- type: cos_sim_pearson
value: 72.21850322994712
- type: cos_sim_spearman
value: 72.28669398117248
- type: euclidean_pearson
value: 73.40082510412948
- type: euclidean_spearman
value: 73.0326539281865
- type: manhattan_pearson
value: 71.8659633964841
- type: manhattan_spearman
value: 71.57817425823303
task:
type: STS
- dataset:
config: it-en
name: MTEB STS17 (it-en)
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
split: test
type: mteb/sts17-crosslingual-sts
metrics:
- type: cos_sim_pearson
value: 75.80921368595645
- type: cos_sim_spearman
value: 77.33209091229315
- type: euclidean_pearson
value: 76.53159540154829
- type: euclidean_spearman
value: 78.17960842810093
- type: manhattan_pearson
value: 76.13530186637601
- type: manhattan_spearman
value: 78.00701437666875
task:
type: STS
- dataset:
config: nl-en
name: MTEB STS17 (nl-en)
revision: af5e6fb845001ecf41f4c1e033ce921939a2a68d
split: test
type: mteb/sts17-crosslingual-sts
metrics:
- type: cos_sim_pearson
value: 74.74980608267349
- type: cos_sim_spearman
value: 75.37597374318821
- type: euclidean_pearson
value: 74.90506081911661
- type: euclidean_spearman
value: 75.30151613124521
- type: manhattan_pearson
value: 74.62642745918002
- type: manhattan_spearman
value: 75.18619716592303
task:
type: STS
- dataset:
config: en
name: MTEB STS22 (en)
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
split: test
type: mteb/sts22-crosslingual-sts
metrics:
- type: cos_sim_pearson
value: 59.632662289205584
- type: cos_sim_spearman
value: 60.938543391610914
- type: euclidean_pearson
value: 62.113200529767056
- type: euclidean_spearman
value: 61.410312633261164
- type: manhattan_pearson
value: 61.75494698945686
- type: manhattan_spearman
value: 60.92726195322362
task:
type: STS
- dataset:
config: de
name: MTEB STS22 (de)
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
split: test
type: mteb/sts22-crosslingual-sts
metrics:
- type: cos_sim_pearson
value: 45.283470551557244
- type: cos_sim_spearman
value: 53.44833015864201
- type: euclidean_pearson
value: 41.17892011120893
- type: euclidean_spearman
value: 53.81441383126767
- type: manhattan_pearson
value: 41.17482200420659
- type: manhattan_spearman
value: 53.82180269276363
task:
type: STS
- dataset:
config: es
name: MTEB STS22 (es)
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
split: test
type: mteb/sts22-crosslingual-sts
metrics:
- type: cos_sim_pearson
value: 60.5069165306236
- type: cos_sim_spearman
value: 66.87803259033826
- type: euclidean_pearson
value: 63.5428979418236
- type: euclidean_spearman
value: 66.9293576586897
- type: manhattan_pearson
value: 63.59789526178922
- type: manhattan_spearman
value: 66.86555009875066
task:
type: STS
- dataset:
config: pl
name: MTEB STS22 (pl)
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
split: test
type: mteb/sts22-crosslingual-sts
metrics:
- type: cos_sim_pearson
value: 28.23026196280264
- type: cos_sim_spearman
value: 35.79397812652861
- type: euclidean_pearson
value: 17.828102102767353
- type: euclidean_spearman
value: 35.721501145568894
- type: manhattan_pearson
value: 17.77134274219677
- type: manhattan_spearman
value: 35.98107902846267
task:
type: STS
- dataset:
config: tr
name: MTEB STS22 (tr)
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
split: test
type: mteb/sts22-crosslingual-sts
metrics:
- type: cos_sim_pearson
value: 56.51946541393812
- type: cos_sim_spearman
value: 63.714686006214485
- type: euclidean_pearson
value: 58.32104651305898
- type: euclidean_spearman
value: 62.237110895702216
- type: manhattan_pearson
value: 58.579416468759185
- type: manhattan_spearman
value: 62.459738981727
task:
type: STS
- dataset:
config: ar
name: MTEB STS22 (ar)
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
split: test
type: mteb/sts22-crosslingual-sts
metrics:
- type: cos_sim_pearson
value: 48.76009839569795
- type: cos_sim_spearman
value: 56.65188431953149
- type: euclidean_pearson
value: 50.997682160915595
- type: euclidean_spearman
value: 55.99910008818135
- type: manhattan_pearson
value: 50.76220659606342
- type: manhattan_spearman
value: 55.517347595391456
task:
type: STS
- dataset:
config: ru
name: MTEB STS22 (ru)
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
split: test
type: mteb/sts22-crosslingual-sts
metrics:
- type: cosine_pearson
value: 50.724322379215934
- type: cosine_spearman
value: 59.90449732164651
- type: euclidean_pearson
value: 50.227545226784024
- type: euclidean_spearman
value: 59.898906527601085
- type: main_score
value: 59.90449732164651
- type: manhattan_pearson
value: 50.21762139819405
- type: manhattan_spearman
value: 59.761039813759
- type: pearson
value: 50.724322379215934
- type: spearman
value: 59.90449732164651
task:
type: STS
- dataset:
config: zh
name: MTEB STS22 (zh)
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
split: test
type: mteb/sts22-crosslingual-sts
metrics:
- type: cos_sim_pearson
value: 54.717524559088005
- type: cos_sim_spearman
value: 66.83570886252286
- type: euclidean_pearson
value: 58.41338625505467
- type: euclidean_spearman
value: 66.68991427704938
- type: manhattan_pearson
value: 58.78638572916807
- type: manhattan_spearman
value: 66.58684161046335
task:
type: STS
- dataset:
config: fr
name: MTEB STS22 (fr)
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
split: test
type: mteb/sts22-crosslingual-sts
metrics:
- type: cos_sim_pearson
value: 73.2962042954962
- type: cos_sim_spearman
value: 76.58255504852025
- type: euclidean_pearson
value: 75.70983192778257
- type: euclidean_spearman
value: 77.4547684870542
- type: manhattan_pearson
value: 75.75565853870485
- type: manhattan_spearman
value: 76.90208974949428
task:
type: STS
- dataset:
config: de-en
name: MTEB STS22 (de-en)
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
split: test
type: mteb/sts22-crosslingual-sts
metrics:
- type: cos_sim_pearson
value: 54.47396266924846
- type: cos_sim_spearman
value: 56.492267162048606
- type: euclidean_pearson
value: 55.998505203070195
- type: euclidean_spearman
value: 56.46447012960222
- type: manhattan_pearson
value: 54.873172394430995
- type: manhattan_spearman
value: 56.58111534551218
task:
type: STS
- dataset:
config: es-en
name: MTEB STS22 (es-en)
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
split: test
type: mteb/sts22-crosslingual-sts
metrics:
- type: cos_sim_pearson
value: 69.87177267688686
- type: cos_sim_spearman
value: 74.57160943395763
- type: euclidean_pearson
value: 70.88330406826788
- type: euclidean_spearman
value: 74.29767636038422
- type: manhattan_pearson
value: 71.38245248369536
- type: manhattan_spearman
value: 74.53102232732175
task:
type: STS
- dataset:
config: it
name: MTEB STS22 (it)
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
split: test
type: mteb/sts22-crosslingual-sts
metrics:
- type: cos_sim_pearson
value: 72.80225656959544
- type: cos_sim_spearman
value: 76.52646173725735
- type: euclidean_pearson
value: 73.95710720200799
- type: euclidean_spearman
value: 76.54040031984111
- type: manhattan_pearson
value: 73.89679971946774
- type: manhattan_spearman
value: 76.60886958161574
task:
type: STS
- dataset:
config: pl-en
name: MTEB STS22 (pl-en)
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
split: test
type: mteb/sts22-crosslingual-sts
metrics:
- type: cos_sim_pearson
value: 70.70844249898789
- type: cos_sim_spearman
value: 72.68571783670241
- type: euclidean_pearson
value: 72.38800772441031
- type: euclidean_spearman
value: 72.86804422703312
- type: manhattan_pearson
value: 71.29840508203515
- type: manhattan_spearman
value: 71.86264441749513
task:
type: STS
- dataset:
config: zh-en
name: MTEB STS22 (zh-en)
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
split: test
type: mteb/sts22-crosslingual-sts
metrics:
- type: cos_sim_pearson
value: 58.647478923935694
- type: cos_sim_spearman
value: 63.74453623540931
- type: euclidean_pearson
value: 59.60138032437505
- type: euclidean_spearman
value: 63.947930832166065
- type: manhattan_pearson
value: 58.59735509491861
- type: manhattan_spearman
value: 62.082503844627404
task:
type: STS
- dataset:
config: es-it
name: MTEB STS22 (es-it)
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
split: test
type: mteb/sts22-crosslingual-sts
metrics:
- type: cos_sim_pearson
value: 65.8722516867162
- type: cos_sim_spearman
value: 71.81208592523012
- type: euclidean_pearson
value: 67.95315252165956
- type: euclidean_spearman
value: 73.00749822046009
- type: manhattan_pearson
value: 68.07884688638924
- type: manhattan_spearman
value: 72.34210325803069
task:
type: STS
- dataset:
config: de-fr
name: MTEB STS22 (de-fr)
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
split: test
type: mteb/sts22-crosslingual-sts
metrics:
- type: cos_sim_pearson
value: 54.5405814240949
- type: cos_sim_spearman
value: 60.56838649023775
- type: euclidean_pearson
value: 53.011731611314104
- type: euclidean_spearman
value: 58.533194841668426
- type: manhattan_pearson
value: 53.623067729338494
- type: manhattan_spearman
value: 58.018756154446926
task:
type: STS
- dataset:
config: de-pl
name: MTEB STS22 (de-pl)
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
split: test
type: mteb/sts22-crosslingual-sts
metrics:
- type: cos_sim_pearson
value: 13.611046866216112
- type: cos_sim_spearman
value: 28.238192909158492
- type: euclidean_pearson
value: 22.16189199885129
- type: euclidean_spearman
value: 35.012895679076564
- type: manhattan_pearson
value: 21.969771178698387
- type: manhattan_spearman
value: 32.456985088607475
task:
type: STS
- dataset:
config: fr-pl
name: MTEB STS22 (fr-pl)
revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
split: test
type: mteb/sts22-crosslingual-sts
metrics:
- type: cos_sim_pearson
value: 74.58077407011655
- type: cos_sim_spearman
value: 84.51542547285167
- type: euclidean_pearson
value: 74.64613843596234
- type: euclidean_spearman
value: 84.51542547285167
- type: manhattan_pearson
value: 75.15335973101396
- type: manhattan_spearman
value: 84.51542547285167
task:
type: STS
- dataset:
config: default
name: MTEB STSBenchmark
revision: b0fddb56ed78048fa8b90373c8a3cfc37b684831
split: test
type: mteb/stsbenchmark-sts
metrics:
- type: cos_sim_pearson
value: 82.0739825531578
- type: cos_sim_spearman
value: 84.01057479311115
- type: euclidean_pearson
value: 83.85453227433344
- type: euclidean_spearman
value: 84.01630226898655
- type: manhattan_pearson
value: 83.75323603028978
- type: manhattan_spearman
value: 83.89677983727685
task:
type: STS
- dataset:
config: default
name: MTEB SciDocsRR
revision: d3c5e1fc0b855ab6097bf1cda04dd73947d7caab
split: test
type: mteb/scidocs-reranking
metrics:
- type: map
value: 78.12945623123957
- type: mrr
value: 93.87738713719106
task:
type: Reranking
- dataset:
config: default
name: MTEB SciFact
revision: None
split: test
type: scifact
metrics:
- type: map_at_1
value: 52.983000000000004
- type: map_at_10
value: 62.946000000000005
- type: map_at_100
value: 63.514
- type: map_at_1000
value: 63.554
- type: map_at_3
value: 60.183
- type: map_at_5
value: 61.672000000000004
- type: mrr_at_1
value: 55.667
- type: mrr_at_10
value: 64.522
- type: mrr_at_100
value: 64.957
- type: mrr_at_1000
value: 64.995
- type: mrr_at_3
value: 62.388999999999996
- type: mrr_at_5
value: 63.639
- type: ndcg_at_1
value: 55.667
- type: ndcg_at_10
value: 67.704
- type: ndcg_at_100
value: 70.299
- type: ndcg_at_1000
value: 71.241
- type: ndcg_at_3
value: 62.866
- type: ndcg_at_5
value: 65.16999999999999
- type: precision_at_1
value: 55.667
- type: precision_at_10
value: 9.033
- type: precision_at_100
value: 1.053
- type: precision_at_1000
value: 0.11299999999999999
- type: precision_at_3
value: 24.444
- type: precision_at_5
value: 16.133
- type: recall_at_1
value: 52.983000000000004
- type: recall_at_10
value: 80.656
- type: recall_at_100
value: 92.5
- type: recall_at_1000
value: 99.667
- type: recall_at_3
value: 67.744
- type: recall_at_5
value: 73.433
task:
type: Retrieval
- dataset:
config: default
name: MTEB SprintDuplicateQuestions
revision: d66bd1f72af766a5cc4b0ca5e00c162f89e8cc46
split: test
type: mteb/sprintduplicatequestions-pairclassification
metrics:
- type: cos_sim_accuracy
value: 99.72772277227723
- type: cos_sim_ap
value: 92.17845897992215
- type: cos_sim_f1
value: 85.9746835443038
- type: cos_sim_precision
value: 87.07692307692308
- type: cos_sim_recall
value: 84.89999999999999
- type: dot_accuracy
value: 99.3039603960396
- type: dot_ap
value: 60.70244020124878
- type: dot_f1
value: 59.92742353551063
- type: dot_precision
value: 62.21743810548978
- type: dot_recall
value: 57.8
- type: euclidean_accuracy
value: 99.71683168316832
- type: euclidean_ap
value: 91.53997039964659
- type: euclidean_f1
value: 84.88372093023257
- type: euclidean_precision
value: 90.02242152466367
- type: euclidean_recall
value: 80.30000000000001
- type: manhattan_accuracy
value: 99.72376237623763
- type: manhattan_ap
value: 91.80756777790289
- type: manhattan_f1
value: 85.48468106479157
- type: manhattan_precision
value: 85.8728557013118
- type: manhattan_recall
value: 85.1
- type: max_accuracy
value: 99.72772277227723
- type: max_ap
value: 92.17845897992215
- type: max_f1
value: 85.9746835443038
task:
type: PairClassification
- dataset:
config: default
name: MTEB StackExchangeClustering
revision: 6cbc1f7b2bc0622f2e39d2c77fa502909748c259
split: test
type: mteb/stackexchange-clustering
metrics:
- type: v_measure
value: 53.52464042600003
task:
type: Clustering
- dataset:
config: default
name: MTEB StackExchangeClusteringP2P
revision: 815ca46b2622cec33ccafc3735d572c266efdb44
split: test
type: mteb/stackexchange-clustering-p2p
metrics:
- type: v_measure
value: 32.071631948736
task:
type: Clustering
- dataset:
config: default
name: MTEB StackOverflowDupQuestions
revision: e185fbe320c72810689fc5848eb6114e1ef5ec69
split: test
type: mteb/stackoverflowdupquestions-reranking
metrics:
- type: map
value: 49.19552407604654
- type: mrr
value: 49.95269130379425
task:
type: Reranking
- dataset:
config: default
name: MTEB SummEval
revision: cda12ad7615edc362dbf25a00fdd61d3b1eaf93c
split: test
type: mteb/summeval
metrics:
- type: cos_sim_pearson
value: 29.345293033095427
- type: cos_sim_spearman
value: 29.976931423258403
- type: dot_pearson
value: 27.047078008958408
- type: dot_spearman
value: 27.75894368380218
task:
type: Summarization
- dataset:
config: default
name: MTEB TRECCOVID
revision: None
split: test
type: trec-covid
metrics:
- type: map_at_1
value: 0.22
- type: map_at_10
value: 1.706
- type: map_at_100
value: 9.634
- type: map_at_1000
value: 23.665
- type: map_at_3
value: 0.5950000000000001
- type: map_at_5
value: 0.95
- type: mrr_at_1
value: 86.0
- type: mrr_at_10
value: 91.8
- type: mrr_at_100
value: 91.8
- type: mrr_at_1000
value: 91.8
- type: mrr_at_3
value: 91.0
- type: mrr_at_5
value: 91.8
- type: ndcg_at_1
value: 80.0
- type: ndcg_at_10
value: 72.573
- type: ndcg_at_100
value: 53.954
- type: ndcg_at_1000
value: 47.760999999999996
- type: ndcg_at_3
value: 76.173
- type: ndcg_at_5
value: 75.264
- type: precision_at_1
value: 86.0
- type: precision_at_10
value: 76.4
- type: precision_at_100
value: 55.50000000000001
- type: precision_at_1000
value: 21.802
- type: precision_at_3
value: 81.333
- type: precision_at_5
value: 80.4
- type: recall_at_1
value: 0.22
- type: recall_at_10
value: 1.925
- type: recall_at_100
value: 12.762
- type: recall_at_1000
value: 44.946000000000005
- type: recall_at_3
value: 0.634
- type: recall_at_5
value: 1.051
task:
type: Retrieval
- dataset:
config: sqi-eng
name: MTEB Tatoeba (sqi-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 91.0
- type: f1
value: 88.55666666666666
- type: precision
value: 87.46166666666667
- type: recall
value: 91.0
task:
type: BitextMining
- dataset:
config: fry-eng
name: MTEB Tatoeba (fry-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 57.22543352601156
- type: f1
value: 51.03220478943021
- type: precision
value: 48.8150289017341
- type: recall
value: 57.22543352601156
task:
type: BitextMining
- dataset:
config: kur-eng
name: MTEB Tatoeba (kur-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 46.58536585365854
- type: f1
value: 39.66870798578116
- type: precision
value: 37.416085946573745
- type: recall
value: 46.58536585365854
task:
type: BitextMining
- dataset:
config: tur-eng
name: MTEB Tatoeba (tur-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 89.7
- type: f1
value: 86.77999999999999
- type: precision
value: 85.45333333333332
- type: recall
value: 89.7
task:
type: BitextMining
- dataset:
config: deu-eng
name: MTEB Tatoeba (deu-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 97.39999999999999
- type: f1
value: 96.58333333333331
- type: precision
value: 96.2
- type: recall
value: 97.39999999999999
task:
type: BitextMining
- dataset:
config: nld-eng
name: MTEB Tatoeba (nld-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 92.4
- type: f1
value: 90.3
- type: precision
value: 89.31666666666668
- type: recall
value: 92.4
task:
type: BitextMining
- dataset:
config: ron-eng
name: MTEB Tatoeba (ron-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 86.9
- type: f1
value: 83.67190476190476
- type: precision
value: 82.23333333333332
- type: recall
value: 86.9
task:
type: BitextMining
- dataset:
config: ang-eng
name: MTEB Tatoeba (ang-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 50.0
- type: f1
value: 42.23229092632078
- type: precision
value: 39.851634683724235
- type: recall
value: 50.0
task:
type: BitextMining
- dataset:
config: ido-eng
name: MTEB Tatoeba (ido-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 76.3
- type: f1
value: 70.86190476190477
- type: precision
value: 68.68777777777777
- type: recall
value: 76.3
task:
type: BitextMining
- dataset:
config: jav-eng
name: MTEB Tatoeba (jav-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 57.073170731707314
- type: f1
value: 50.658958927251604
- type: precision
value: 48.26480836236933
- type: recall
value: 57.073170731707314
task:
type: BitextMining
- dataset:
config: isl-eng
name: MTEB Tatoeba (isl-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 68.2
- type: f1
value: 62.156507936507936
- type: precision
value: 59.84964285714286
- type: recall
value: 68.2
task:
type: BitextMining
- dataset:
config: slv-eng
name: MTEB Tatoeba (slv-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 77.52126366950182
- type: f1
value: 72.8496210148701
- type: precision
value: 70.92171498003819
- type: recall
value: 77.52126366950182
task:
type: BitextMining
- dataset:
config: cym-eng
name: MTEB Tatoeba (cym-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 70.78260869565217
- type: f1
value: 65.32422360248447
- type: precision
value: 63.063067367415194
- type: recall
value: 70.78260869565217
task:
type: BitextMining
- dataset:
config: kaz-eng
name: MTEB Tatoeba (kaz-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 78.43478260869566
- type: f1
value: 73.02608695652172
- type: precision
value: 70.63768115942028
- type: recall
value: 78.43478260869566
task:
type: BitextMining
- dataset:
config: est-eng
name: MTEB Tatoeba (est-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 60.9
- type: f1
value: 55.309753694581275
- type: precision
value: 53.130476190476195
- type: recall
value: 60.9
task:
type: BitextMining
- dataset:
config: heb-eng
name: MTEB Tatoeba (heb-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 72.89999999999999
- type: f1
value: 67.92023809523809
- type: precision
value: 65.82595238095237
- type: recall
value: 72.89999999999999
task:
type: BitextMining
- dataset:
config: gla-eng
name: MTEB Tatoeba (gla-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 46.80337756332931
- type: f1
value: 39.42174900558496
- type: precision
value: 36.97101116280851
- type: recall
value: 46.80337756332931
task:
type: BitextMining
- dataset:
config: mar-eng
name: MTEB Tatoeba (mar-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 89.8
- type: f1
value: 86.79
- type: precision
value: 85.375
- type: recall
value: 89.8
task:
type: BitextMining
- dataset:
config: lat-eng
name: MTEB Tatoeba (lat-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 47.199999999999996
- type: f1
value: 39.95484348984349
- type: precision
value: 37.561071428571424
- type: recall
value: 47.199999999999996
task:
type: BitextMining
- dataset:
config: bel-eng
name: MTEB Tatoeba (bel-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 87.8
- type: f1
value: 84.68190476190475
- type: precision
value: 83.275
- type: recall
value: 87.8
task:
type: BitextMining
- dataset:
config: pms-eng
name: MTEB Tatoeba (pms-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 48.76190476190476
- type: f1
value: 42.14965986394558
- type: precision
value: 39.96743626743626
- type: recall
value: 48.76190476190476
task:
type: BitextMining
- dataset:
config: gle-eng
name: MTEB Tatoeba (gle-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 66.10000000000001
- type: f1
value: 59.58580086580086
- type: precision
value: 57.150238095238095
- type: recall
value: 66.10000000000001
task:
type: BitextMining
- dataset:
config: pes-eng
name: MTEB Tatoeba (pes-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 87.3
- type: f1
value: 84.0
- type: precision
value: 82.48666666666666
- type: recall
value: 87.3
task:
type: BitextMining
- dataset:
config: nob-eng
name: MTEB Tatoeba (nob-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 90.4
- type: f1
value: 87.79523809523809
- type: precision
value: 86.6
- type: recall
value: 90.4
task:
type: BitextMining
- dataset:
config: bul-eng
name: MTEB Tatoeba (bul-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 87.0
- type: f1
value: 83.81
- type: precision
value: 82.36666666666666
- type: recall
value: 87.0
task:
type: BitextMining
- dataset:
config: cbk-eng
name: MTEB Tatoeba (cbk-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 63.9
- type: f1
value: 57.76533189033189
- type: precision
value: 55.50595238095239
- type: recall
value: 63.9
task:
type: BitextMining
- dataset:
config: hun-eng
name: MTEB Tatoeba (hun-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 76.1
- type: f1
value: 71.83690476190478
- type: precision
value: 70.04928571428573
- type: recall
value: 76.1
task:
type: BitextMining
- dataset:
config: uig-eng
name: MTEB Tatoeba (uig-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 66.3
- type: f1
value: 59.32626984126984
- type: precision
value: 56.62535714285713
- type: recall
value: 66.3
task:
type: BitextMining
- dataset:
config: rus-eng
name: MTEB Tatoeba (rus-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 92.10000000000001
- type: f1
value: 89.76666666666667
- type: main_score
value: 89.76666666666667
- type: precision
value: 88.64999999999999
- type: recall
value: 92.10000000000001
task:
type: BitextMining
- dataset:
config: spa-eng
name: MTEB Tatoeba (spa-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 93.10000000000001
- type: f1
value: 91.10000000000001
- type: precision
value: 90.16666666666666
- type: recall
value: 93.10000000000001
task:
type: BitextMining
- dataset:
config: hye-eng
name: MTEB Tatoeba (hye-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 85.71428571428571
- type: f1
value: 82.29142600436403
- type: precision
value: 80.8076626877166
- type: recall
value: 85.71428571428571
task:
type: BitextMining
- dataset:
config: tel-eng
name: MTEB Tatoeba (tel-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 88.88888888888889
- type: f1
value: 85.7834757834758
- type: precision
value: 84.43732193732193
- type: recall
value: 88.88888888888889
task:
type: BitextMining
- dataset:
config: afr-eng
name: MTEB Tatoeba (afr-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 88.5
- type: f1
value: 85.67190476190476
- type: precision
value: 84.43333333333332
- type: recall
value: 88.5
task:
type: BitextMining
- dataset:
config: mon-eng
name: MTEB Tatoeba (mon-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 82.72727272727273
- type: f1
value: 78.21969696969695
- type: precision
value: 76.18181818181819
- type: recall
value: 82.72727272727273
task:
type: BitextMining
- dataset:
config: arz-eng
name: MTEB Tatoeba (arz-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 61.0062893081761
- type: f1
value: 55.13976240391334
- type: precision
value: 52.92112499659669
- type: recall
value: 61.0062893081761
task:
type: BitextMining
- dataset:
config: hrv-eng
name: MTEB Tatoeba (hrv-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 89.5
- type: f1
value: 86.86666666666666
- type: precision
value: 85.69166666666668
- type: recall
value: 89.5
task:
type: BitextMining
- dataset:
config: nov-eng
name: MTEB Tatoeba (nov-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 73.54085603112841
- type: f1
value: 68.56031128404669
- type: precision
value: 66.53047989623866
- type: recall
value: 73.54085603112841
task:
type: BitextMining
- dataset:
config: gsw-eng
name: MTEB Tatoeba (gsw-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 43.58974358974359
- type: f1
value: 36.45299145299145
- type: precision
value: 33.81155881155882
- type: recall
value: 43.58974358974359
task:
type: BitextMining
- dataset:
config: nds-eng
name: MTEB Tatoeba (nds-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 59.599999999999994
- type: f1
value: 53.264689754689755
- type: precision
value: 50.869166666666665
- type: recall
value: 59.599999999999994
task:
type: BitextMining
- dataset:
config: ukr-eng
name: MTEB Tatoeba (ukr-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 85.2
- type: f1
value: 81.61666666666665
- type: precision
value: 80.02833333333335
- type: recall
value: 85.2
task:
type: BitextMining
- dataset:
config: uzb-eng
name: MTEB Tatoeba (uzb-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 63.78504672897196
- type: f1
value: 58.00029669188548
- type: precision
value: 55.815809968847354
- type: recall
value: 63.78504672897196
task:
type: BitextMining
- dataset:
config: lit-eng
name: MTEB Tatoeba (lit-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 66.5
- type: f1
value: 61.518333333333345
- type: precision
value: 59.622363699102834
- type: recall
value: 66.5
task:
type: BitextMining
- dataset:
config: ina-eng
name: MTEB Tatoeba (ina-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 88.6
- type: f1
value: 85.60222222222221
- type: precision
value: 84.27916666666665
- type: recall
value: 88.6
task:
type: BitextMining
- dataset:
config: lfn-eng
name: MTEB Tatoeba (lfn-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 58.699999999999996
- type: f1
value: 52.732375957375965
- type: precision
value: 50.63214035964035
- type: recall
value: 58.699999999999996
task:
type: BitextMining
- dataset:
config: zsm-eng
name: MTEB Tatoeba (zsm-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 92.10000000000001
- type: f1
value: 89.99666666666667
- type: precision
value: 89.03333333333333
- type: recall
value: 92.10000000000001
task:
type: BitextMining
- dataset:
config: ita-eng
name: MTEB Tatoeba (ita-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 90.10000000000001
- type: f1
value: 87.55666666666667
- type: precision
value: 86.36166666666668
- type: recall
value: 90.10000000000001
task:
type: BitextMining
- dataset:
config: cmn-eng
name: MTEB Tatoeba (cmn-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 91.4
- type: f1
value: 88.89000000000001
- type: precision
value: 87.71166666666666
- type: recall
value: 91.4
task:
type: BitextMining
- dataset:
config: lvs-eng
name: MTEB Tatoeba (lvs-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 65.7
- type: f1
value: 60.67427750410509
- type: precision
value: 58.71785714285714
- type: recall
value: 65.7
task:
type: BitextMining
- dataset:
config: glg-eng
name: MTEB Tatoeba (glg-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 85.39999999999999
- type: f1
value: 81.93190476190475
- type: precision
value: 80.37833333333333
- type: recall
value: 85.39999999999999
task:
type: BitextMining
- dataset:
config: ceb-eng
name: MTEB Tatoeba (ceb-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 47.833333333333336
- type: f1
value: 42.006625781625786
- type: precision
value: 40.077380952380956
- type: recall
value: 47.833333333333336
task:
type: BitextMining
- dataset:
config: bre-eng
name: MTEB Tatoeba (bre-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 10.4
- type: f1
value: 8.24465007215007
- type: precision
value: 7.664597069597071
- type: recall
value: 10.4
task:
type: BitextMining
- dataset:
config: ben-eng
name: MTEB Tatoeba (ben-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 82.6
- type: f1
value: 77.76333333333334
- type: precision
value: 75.57833333333332
- type: recall
value: 82.6
task:
type: BitextMining
- dataset:
config: swg-eng
name: MTEB Tatoeba (swg-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 52.67857142857143
- type: f1
value: 44.302721088435376
- type: precision
value: 41.49801587301587
- type: recall
value: 52.67857142857143
task:
type: BitextMining
- dataset:
config: arq-eng
name: MTEB Tatoeba (arq-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 28.3205268935236
- type: f1
value: 22.426666605171157
- type: precision
value: 20.685900116470915
- type: recall
value: 28.3205268935236
task:
type: BitextMining
- dataset:
config: kab-eng
name: MTEB Tatoeba (kab-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 22.7
- type: f1
value: 17.833970473970474
- type: precision
value: 16.407335164835164
- type: recall
value: 22.7
task:
type: BitextMining
- dataset:
config: fra-eng
name: MTEB Tatoeba (fra-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 92.2
- type: f1
value: 89.92999999999999
- type: precision
value: 88.87
- type: recall
value: 92.2
task:
type: BitextMining
- dataset:
config: por-eng
name: MTEB Tatoeba (por-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 91.4
- type: f1
value: 89.25
- type: precision
value: 88.21666666666667
- type: recall
value: 91.4
task:
type: BitextMining
- dataset:
config: tat-eng
name: MTEB Tatoeba (tat-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 69.19999999999999
- type: f1
value: 63.38269841269841
- type: precision
value: 61.14773809523809
- type: recall
value: 69.19999999999999
task:
type: BitextMining
- dataset:
config: oci-eng
name: MTEB Tatoeba (oci-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 48.8
- type: f1
value: 42.839915639915645
- type: precision
value: 40.770287114845935
- type: recall
value: 48.8
task:
type: BitextMining
- dataset:
config: pol-eng
name: MTEB Tatoeba (pol-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 88.8
- type: f1
value: 85.90666666666668
- type: precision
value: 84.54166666666666
- type: recall
value: 88.8
task:
type: BitextMining
- dataset:
config: war-eng
name: MTEB Tatoeba (war-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 46.6
- type: f1
value: 40.85892920804686
- type: precision
value: 38.838223114604695
- type: recall
value: 46.6
task:
type: BitextMining
- dataset:
config: aze-eng
name: MTEB Tatoeba (aze-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 84.0
- type: f1
value: 80.14190476190475
- type: precision
value: 78.45333333333333
- type: recall
value: 84.0
task:
type: BitextMining
- dataset:
config: vie-eng
name: MTEB Tatoeba (vie-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 90.5
- type: f1
value: 87.78333333333333
- type: precision
value: 86.5
- type: recall
value: 90.5
task:
type: BitextMining
- dataset:
config: nno-eng
name: MTEB Tatoeba (nno-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 74.5
- type: f1
value: 69.48397546897547
- type: precision
value: 67.51869047619049
- type: recall
value: 74.5
task:
type: BitextMining
- dataset:
config: cha-eng
name: MTEB Tatoeba (cha-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 32.846715328467155
- type: f1
value: 27.828177499710343
- type: precision
value: 26.63451511991658
- type: recall
value: 32.846715328467155
task:
type: BitextMining
- dataset:
config: mhr-eng
name: MTEB Tatoeba (mhr-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 8.0
- type: f1
value: 6.07664116764988
- type: precision
value: 5.544177607179943
- type: recall
value: 8.0
task:
type: BitextMining
- dataset:
config: dan-eng
name: MTEB Tatoeba (dan-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 87.6
- type: f1
value: 84.38555555555554
- type: precision
value: 82.91583333333334
- type: recall
value: 87.6
task:
type: BitextMining
- dataset:
config: ell-eng
name: MTEB Tatoeba (ell-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 87.5
- type: f1
value: 84.08333333333331
- type: precision
value: 82.47333333333333
- type: recall
value: 87.5
task:
type: BitextMining
- dataset:
config: amh-eng
name: MTEB Tatoeba (amh-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 80.95238095238095
- type: f1
value: 76.13095238095238
- type: precision
value: 74.05753968253967
- type: recall
value: 80.95238095238095
task:
type: BitextMining
- dataset:
config: pam-eng
name: MTEB Tatoeba (pam-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 8.799999999999999
- type: f1
value: 6.971422975172975
- type: precision
value: 6.557814916172301
- type: recall
value: 8.799999999999999
task:
type: BitextMining
- dataset:
config: hsb-eng
name: MTEB Tatoeba (hsb-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 44.099378881987576
- type: f1
value: 37.01649742022413
- type: precision
value: 34.69420618488942
- type: recall
value: 44.099378881987576
task:
type: BitextMining
- dataset:
config: srp-eng
name: MTEB Tatoeba (srp-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 84.3
- type: f1
value: 80.32666666666667
- type: precision
value: 78.60666666666665
- type: recall
value: 84.3
task:
type: BitextMining
- dataset:
config: epo-eng
name: MTEB Tatoeba (epo-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 92.5
- type: f1
value: 90.49666666666666
- type: precision
value: 89.56666666666668
- type: recall
value: 92.5
task:
type: BitextMining
- dataset:
config: kzj-eng
name: MTEB Tatoeba (kzj-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 10.0
- type: f1
value: 8.268423529875141
- type: precision
value: 7.878118605532398
- type: recall
value: 10.0
task:
type: BitextMining
- dataset:
config: awa-eng
name: MTEB Tatoeba (awa-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 79.22077922077922
- type: f1
value: 74.27128427128426
- type: precision
value: 72.28715728715729
- type: recall
value: 79.22077922077922
task:
type: BitextMining
- dataset:
config: fao-eng
name: MTEB Tatoeba (fao-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 65.64885496183206
- type: f1
value: 58.87495456197747
- type: precision
value: 55.992366412213734
- type: recall
value: 65.64885496183206
task:
type: BitextMining
- dataset:
config: mal-eng
name: MTEB Tatoeba (mal-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 96.06986899563319
- type: f1
value: 94.78408539543909
- type: precision
value: 94.15332362930616
- type: recall
value: 96.06986899563319
task:
type: BitextMining
- dataset:
config: ile-eng
name: MTEB Tatoeba (ile-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 77.2
- type: f1
value: 71.72571428571428
- type: precision
value: 69.41000000000001
- type: recall
value: 77.2
task:
type: BitextMining
- dataset:
config: bos-eng
name: MTEB Tatoeba (bos-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 86.4406779661017
- type: f1
value: 83.2391713747646
- type: precision
value: 81.74199623352166
- type: recall
value: 86.4406779661017
task:
type: BitextMining
- dataset:
config: cor-eng
name: MTEB Tatoeba (cor-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 8.4
- type: f1
value: 6.017828743398003
- type: precision
value: 5.4829865484756795
- type: recall
value: 8.4
task:
type: BitextMining
- dataset:
config: cat-eng
name: MTEB Tatoeba (cat-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 83.5
- type: f1
value: 79.74833333333333
- type: precision
value: 78.04837662337664
- type: recall
value: 83.5
task:
type: BitextMining
- dataset:
config: eus-eng
name: MTEB Tatoeba (eus-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 60.4
- type: f1
value: 54.467301587301584
- type: precision
value: 52.23242424242424
- type: recall
value: 60.4
task:
type: BitextMining
- dataset:
config: yue-eng
name: MTEB Tatoeba (yue-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 74.9
- type: f1
value: 69.68699134199134
- type: precision
value: 67.59873015873016
- type: recall
value: 74.9
task:
type: BitextMining
- dataset:
config: swe-eng
name: MTEB Tatoeba (swe-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 88.0
- type: f1
value: 84.9652380952381
- type: precision
value: 83.66166666666666
- type: recall
value: 88.0
task:
type: BitextMining
- dataset:
config: dtp-eng
name: MTEB Tatoeba (dtp-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 9.1
- type: f1
value: 7.681244588744588
- type: precision
value: 7.370043290043291
- type: recall
value: 9.1
task:
type: BitextMining
- dataset:
config: kat-eng
name: MTEB Tatoeba (kat-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 80.9651474530831
- type: f1
value: 76.84220605132133
- type: precision
value: 75.19606398962966
- type: recall
value: 80.9651474530831
task:
type: BitextMining
- dataset:
config: jpn-eng
name: MTEB Tatoeba (jpn-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 86.9
- type: f1
value: 83.705
- type: precision
value: 82.3120634920635
- type: recall
value: 86.9
task:
type: BitextMining
- dataset:
config: csb-eng
name: MTEB Tatoeba (csb-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 29.64426877470356
- type: f1
value: 23.98763072676116
- type: precision
value: 22.506399397703746
- type: recall
value: 29.64426877470356
task:
type: BitextMining
- dataset:
config: xho-eng
name: MTEB Tatoeba (xho-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 70.4225352112676
- type: f1
value: 62.84037558685445
- type: precision
value: 59.56572769953053
- type: recall
value: 70.4225352112676
task:
type: BitextMining
- dataset:
config: orv-eng
name: MTEB Tatoeba (orv-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 19.64071856287425
- type: f1
value: 15.125271011207756
- type: precision
value: 13.865019261197494
- type: recall
value: 19.64071856287425
task:
type: BitextMining
- dataset:
config: ind-eng
name: MTEB Tatoeba (ind-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 90.2
- type: f1
value: 87.80666666666666
- type: precision
value: 86.70833333333331
- type: recall
value: 90.2
task:
type: BitextMining
- dataset:
config: tuk-eng
name: MTEB Tatoeba (tuk-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 23.15270935960591
- type: f1
value: 18.407224958949097
- type: precision
value: 16.982385430661292
- type: recall
value: 23.15270935960591
task:
type: BitextMining
- dataset:
config: max-eng
name: MTEB Tatoeba (max-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 55.98591549295775
- type: f1
value: 49.94718309859154
- type: precision
value: 47.77864154624717
- type: recall
value: 55.98591549295775
task:
type: BitextMining
- dataset:
config: swh-eng
name: MTEB Tatoeba (swh-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 73.07692307692307
- type: f1
value: 66.74358974358974
- type: precision
value: 64.06837606837607
- type: recall
value: 73.07692307692307
task:
type: BitextMining
- dataset:
config: hin-eng
name: MTEB Tatoeba (hin-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 94.89999999999999
- type: f1
value: 93.25
- type: precision
value: 92.43333333333332
- type: recall
value: 94.89999999999999
task:
type: BitextMining
- dataset:
config: dsb-eng
name: MTEB Tatoeba (dsb-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 37.78705636743215
- type: f1
value: 31.63899658680452
- type: precision
value: 29.72264397629742
- type: recall
value: 37.78705636743215
task:
type: BitextMining
- dataset:
config: ber-eng
name: MTEB Tatoeba (ber-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 21.6
- type: f1
value: 16.91697302697303
- type: precision
value: 15.71225147075147
- type: recall
value: 21.6
task:
type: BitextMining
- dataset:
config: tam-eng
name: MTEB Tatoeba (tam-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 85.01628664495115
- type: f1
value: 81.38514037536838
- type: precision
value: 79.83170466883823
- type: recall
value: 85.01628664495115
task:
type: BitextMining
- dataset:
config: slk-eng
name: MTEB Tatoeba (slk-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 83.39999999999999
- type: f1
value: 79.96380952380952
- type: precision
value: 78.48333333333333
- type: recall
value: 83.39999999999999
task:
type: BitextMining
- dataset:
config: tgl-eng
name: MTEB Tatoeba (tgl-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 83.2
- type: f1
value: 79.26190476190476
- type: precision
value: 77.58833333333334
- type: recall
value: 83.2
task:
type: BitextMining
- dataset:
config: ast-eng
name: MTEB Tatoeba (ast-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 75.59055118110236
- type: f1
value: 71.66854143232096
- type: precision
value: 70.30183727034121
- type: recall
value: 75.59055118110236
task:
type: BitextMining
- dataset:
config: mkd-eng
name: MTEB Tatoeba (mkd-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 65.5
- type: f1
value: 59.26095238095238
- type: precision
value: 56.81909090909092
- type: recall
value: 65.5
task:
type: BitextMining
- dataset:
config: khm-eng
name: MTEB Tatoeba (khm-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 55.26315789473685
- type: f1
value: 47.986523325858506
- type: precision
value: 45.33950006595436
- type: recall
value: 55.26315789473685
task:
type: BitextMining
- dataset:
config: ces-eng
name: MTEB Tatoeba (ces-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 82.89999999999999
- type: f1
value: 78.835
- type: precision
value: 77.04761904761905
- type: recall
value: 82.89999999999999
task:
type: BitextMining
- dataset:
config: tzl-eng
name: MTEB Tatoeba (tzl-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 43.269230769230774
- type: f1
value: 36.20421245421245
- type: precision
value: 33.57371794871795
- type: recall
value: 43.269230769230774
task:
type: BitextMining
- dataset:
config: urd-eng
name: MTEB Tatoeba (urd-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 88.0
- type: f1
value: 84.70666666666666
- type: precision
value: 83.23166666666665
- type: recall
value: 88.0
task:
type: BitextMining
- dataset:
config: ara-eng
name: MTEB Tatoeba (ara-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 77.4
- type: f1
value: 72.54666666666667
- type: precision
value: 70.54318181818181
- type: recall
value: 77.4
task:
type: BitextMining
- dataset:
config: kor-eng
name: MTEB Tatoeba (kor-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 78.60000000000001
- type: f1
value: 74.1588888888889
- type: precision
value: 72.30250000000001
- type: recall
value: 78.60000000000001
task:
type: BitextMining
- dataset:
config: yid-eng
name: MTEB Tatoeba (yid-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 72.40566037735849
- type: f1
value: 66.82587328813744
- type: precision
value: 64.75039308176099
- type: recall
value: 72.40566037735849
task:
type: BitextMining
- dataset:
config: fin-eng
name: MTEB Tatoeba (fin-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 73.8
- type: f1
value: 68.56357142857144
- type: precision
value: 66.3178822055138
- type: recall
value: 73.8
task:
type: BitextMining
- dataset:
config: tha-eng
name: MTEB Tatoeba (tha-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 91.78832116788321
- type: f1
value: 89.3552311435523
- type: precision
value: 88.20559610705597
- type: recall
value: 91.78832116788321
task:
type: BitextMining
- dataset:
config: wuu-eng
name: MTEB Tatoeba (wuu-eng)
revision: 9080400076fbadbb4c4dcb136ff4eddc40b42553
split: test
type: mteb/tatoeba-bitext-mining
metrics:
- type: accuracy
value: 74.3
- type: f1
value: 69.05085581085581
- type: precision
value: 66.955
- type: recall
value: 74.3
task:
type: BitextMining
- dataset:
config: default
name: MTEB Touche2020
revision: None
split: test
type: webis-touche2020
metrics:
- type: map_at_1
value: 2.896
- type: map_at_10
value: 8.993
- type: map_at_100
value: 14.133999999999999
- type: map_at_1000
value: 15.668000000000001
- type: map_at_3
value: 5.862
- type: map_at_5
value: 7.17
- type: mrr_at_1
value: 34.694
- type: mrr_at_10
value: 42.931000000000004
- type: mrr_at_100
value: 44.81
- type: mrr_at_1000
value: 44.81
- type: mrr_at_3
value: 38.435
- type: mrr_at_5
value: 41.701
- type: ndcg_at_1
value: 31.633
- type: ndcg_at_10
value: 21.163
- type: ndcg_at_100
value: 33.306000000000004
- type: ndcg_at_1000
value: 45.275999999999996
- type: ndcg_at_3
value: 25.685999999999996
- type: ndcg_at_5
value: 23.732
- type: precision_at_1
value: 34.694
- type: precision_at_10
value: 17.755000000000003
- type: precision_at_100
value: 6.938999999999999
- type: precision_at_1000
value: 1.48
- type: precision_at_3
value: 25.85
- type: precision_at_5
value: 23.265
- type: recall_at_1
value: 2.896
- type: recall_at_10
value: 13.333999999999998
- type: recall_at_100
value: 43.517
- type: recall_at_1000
value: 79.836
- type: recall_at_3
value: 6.306000000000001
- type: recall_at_5
value: 8.825
task:
type: Retrieval
- dataset:
config: default
name: MTEB ToxicConversationsClassification
revision: d7c0de2777da35d6aae2200a62c6e0e5af397c4c
split: test
type: mteb/toxic_conversations_50k
metrics:
- type: accuracy
value: 69.3874
- type: ap
value: 13.829909072469423
- type: f1
value: 53.54534203543492
task:
type: Classification
- dataset:
config: default
name: MTEB TweetSentimentExtractionClassification
revision: d604517c81ca91fe16a244d1248fc021f9ecee7a
split: test
type: mteb/tweet_sentiment_extraction
metrics:
- type: accuracy
value: 62.62026032823995
- type: f1
value: 62.85251350485221
task:
type: Classification
- dataset:
config: default
name: MTEB TwentyNewsgroupsClustering
revision: 6125ec4e24fa026cec8a478383ee943acfbd5449
split: test
type: mteb/twentynewsgroups-clustering
metrics:
- type: v_measure
value: 33.21527881409797
task:
type: Clustering
- dataset:
config: default
name: MTEB TwitterSemEval2015
revision: 70970daeab8776df92f5ea462b6173c0b46fd2d1
split: test
type: mteb/twittersemeval2015-pairclassification
metrics:
- type: cos_sim_accuracy
value: 84.97943613280086
- type: cos_sim_ap
value: 70.75454316885921
- type: cos_sim_f1
value: 65.38274012676743
- type: cos_sim_precision
value: 60.761214318078835
- type: cos_sim_recall
value: 70.76517150395777
- type: dot_accuracy
value: 79.0546581629612
- type: dot_ap
value: 47.3197121792147
- type: dot_f1
value: 49.20106524633821
- type: dot_precision
value: 42.45499808502489
- type: dot_recall
value: 58.49604221635884
- type: euclidean_accuracy
value: 85.08076533349228
- type: euclidean_ap
value: 70.95016106374474
- type: euclidean_f1
value: 65.43987900176455
- type: euclidean_precision
value: 62.64478764478765
- type: euclidean_recall
value: 68.49604221635884
- type: manhattan_accuracy
value: 84.93771234428085
- type: manhattan_ap
value: 70.63668388755362
- type: manhattan_f1
value: 65.23895401262398
- type: manhattan_precision
value: 56.946084218811485
- type: manhattan_recall
value: 76.35883905013192
- type: max_accuracy
value: 85.08076533349228
- type: max_ap
value: 70.95016106374474
- type: max_f1
value: 65.43987900176455
task:
type: PairClassification
- dataset:
config: default
name: MTEB TwitterURLCorpus
revision: 8b6510b0b1fa4e4c4f879467980e9be563ec1cdf
split: test
type: mteb/twitterurlcorpus-pairclassification
metrics:
- type: cos_sim_accuracy
value: 88.69096130709822
- type: cos_sim_ap
value: 84.82526278228542
- type: cos_sim_f1
value: 77.65485060585536
- type: cos_sim_precision
value: 75.94582658619167
- type: cos_sim_recall
value: 79.44256236526024
- type: dot_accuracy
value: 80.97954748321496
- type: dot_ap
value: 64.81642914145866
- type: dot_f1
value: 60.631996987229975
- type: dot_precision
value: 54.5897293631712
- type: dot_recall
value: 68.17831844779796
- type: euclidean_accuracy
value: 88.6987231730508
- type: euclidean_ap
value: 84.80003825477253
- type: euclidean_f1
value: 77.67194179854496
- type: euclidean_precision
value: 75.7128235122094
- type: euclidean_recall
value: 79.73514012935017
- type: manhattan_accuracy
value: 88.62692591298949
- type: manhattan_ap
value: 84.80451408255276
- type: manhattan_f1
value: 77.69888949572183
- type: manhattan_precision
value: 73.70311528631622
- type: manhattan_recall
value: 82.15275639051433
- type: max_accuracy
value: 88.6987231730508
- type: max_ap
value: 84.82526278228542
- type: max_f1
value: 77.69888949572183
task:
type: PairClassification
- dataset:
config: ru-en
name: MTEB BUCC.v2 (ru-en)
revision: 1739dc11ffe9b7bfccd7f3d585aeb4c544fc6677
split: test
type: mteb/bucc-bitext-mining
metrics:
- type: accuracy
value: 95.72566678212678
- type: f1
value: 94.42443135896548
- type: main_score
value: 94.42443135896548
- type: precision
value: 93.80868260016165
- type: recall
value: 95.72566678212678
task:
type: BitextMining
- dataset:
config: rus_Cyrl-rus_Cyrl
name: MTEB BelebeleRetrieval (rus_Cyrl-rus_Cyrl)
revision: 75b399394a9803252cfec289d103de462763db7c
split: test
type: facebook/belebele
metrics:
- type: main_score
value: 92.23599999999999
- type: map_at_1
value: 87.111
- type: map_at_10
value: 90.717
- type: map_at_100
value: 90.879
- type: map_at_1000
value: 90.881
- type: map_at_20
value: 90.849
- type: map_at_3
value: 90.074
- type: map_at_5
value: 90.535
- type: mrr_at_1
value: 87.1111111111111
- type: mrr_at_10
value: 90.7173721340388
- type: mrr_at_100
value: 90.87859682638407
- type: mrr_at_1000
value: 90.88093553612326
- type: mrr_at_20
value: 90.84863516113515
- type: mrr_at_3
value: 90.07407407407409
- type: mrr_at_5
value: 90.53518518518521
- type: nauc_map_at_1000_diff1
value: 92.37373187280554
- type: nauc_map_at_1000_max
value: 79.90465445423249
- type: nauc_map_at_1000_std
value: -0.6220290556185463
- type: nauc_map_at_100_diff1
value: 92.37386697345335
- type: nauc_map_at_100_max
value: 79.90991577223959
- type: nauc_map_at_100_std
value: -0.602247514642845
- type: nauc_map_at_10_diff1
value: 92.30907447072467
- type: nauc_map_at_10_max
value: 79.86831935337598
- type: nauc_map_at_10_std
value: -0.7455191860719699
- type: nauc_map_at_1_diff1
value: 93.29828518358822
- type: nauc_map_at_1_max
value: 78.69539619887887
- type: nauc_map_at_1_std
value: -4.097150817605763
- type: nauc_map_at_20_diff1
value: 92.38414149703077
- type: nauc_map_at_20_max
value: 79.94789814504661
- type: nauc_map_at_20_std
value: -0.3928031130400773
- type: nauc_map_at_3_diff1
value: 92.21688899306734
- type: nauc_map_at_3_max
value: 80.34586671780885
- type: nauc_map_at_3_std
value: 0.24088319695435909
- type: nauc_map_at_5_diff1
value: 92.27931726042982
- type: nauc_map_at_5_max
value: 79.99198834003367
- type: nauc_map_at_5_std
value: -0.6296366922840796
- type: nauc_mrr_at_1000_diff1
value: 92.37373187280554
- type: nauc_mrr_at_1000_max
value: 79.90465445423249
- type: nauc_mrr_at_1000_std
value: -0.6220290556185463
- type: nauc_mrr_at_100_diff1
value: 92.37386697345335
- type: nauc_mrr_at_100_max
value: 79.90991577223959
- type: nauc_mrr_at_100_std
value: -0.602247514642845
- type: nauc_mrr_at_10_diff1
value: 92.30907447072467
- type: nauc_mrr_at_10_max
value: 79.86831935337598
- type: nauc_mrr_at_10_std
value: -0.7455191860719699
- type: nauc_mrr_at_1_diff1
value: 93.29828518358822
- type: nauc_mrr_at_1_max
value: 78.69539619887887
- type: nauc_mrr_at_1_std
value: -4.097150817605763
- type: nauc_mrr_at_20_diff1
value: 92.38414149703077
- type: nauc_mrr_at_20_max
value: 79.94789814504661
- type: nauc_mrr_at_20_std
value: -0.3928031130400773
- type: nauc_mrr_at_3_diff1
value: 92.21688899306734
- type: nauc_mrr_at_3_max
value: 80.34586671780885
- type: nauc_mrr_at_3_std
value: 0.24088319695435909
- type: nauc_mrr_at_5_diff1
value: 92.27931726042982
- type: nauc_mrr_at_5_max
value: 79.99198834003367
- type: nauc_mrr_at_5_std
value: -0.6296366922840796
- type: nauc_ndcg_at_1000_diff1
value: 92.30526497646306
- type: nauc_ndcg_at_1000_max
value: 80.12734537480418
- type: nauc_ndcg_at_1000_std
value: 0.22849408935578744
- type: nauc_ndcg_at_100_diff1
value: 92.31347123202318
- type: nauc_ndcg_at_100_max
value: 80.29207038703142
- type: nauc_ndcg_at_100_std
value: 0.816825944406239
- type: nauc_ndcg_at_10_diff1
value: 92.05430189845808
- type: nauc_ndcg_at_10_max
value: 80.16515667442968
- type: nauc_ndcg_at_10_std
value: 0.7486447532544893
- type: nauc_ndcg_at_1_diff1
value: 93.29828518358822
- type: nauc_ndcg_at_1_max
value: 78.69539619887887
- type: nauc_ndcg_at_1_std
value: -4.097150817605763
- type: nauc_ndcg_at_20_diff1
value: 92.40147868825079
- type: nauc_ndcg_at_20_max
value: 80.5117307181802
- type: nauc_ndcg_at_20_std
value: 2.0431351539517033
- type: nauc_ndcg_at_3_diff1
value: 91.88894444422789
- type: nauc_ndcg_at_3_max
value: 81.09256084196045
- type: nauc_ndcg_at_3_std
value: 2.422705909643621
- type: nauc_ndcg_at_5_diff1
value: 91.99711052955728
- type: nauc_ndcg_at_5_max
value: 80.46996334573979
- type: nauc_ndcg_at_5_std
value: 0.9086986899040708
- type: nauc_precision_at_1000_diff1
value: .nan
- type: nauc_precision_at_1000_max
value: .nan
- type: nauc_precision_at_1000_std
value: .nan
- type: nauc_precision_at_100_diff1
value: 93.46405228758012
- type: nauc_precision_at_100_max
value: 100.0
- type: nauc_precision_at_100_std
value: 70.71661998132774
- type: nauc_precision_at_10_diff1
value: 90.13938908896874
- type: nauc_precision_at_10_max
value: 82.21121782046167
- type: nauc_precision_at_10_std
value: 13.075230092036083
- type: nauc_precision_at_1_diff1
value: 93.29828518358822
- type: nauc_precision_at_1_max
value: 78.69539619887887
- type: nauc_precision_at_1_std
value: -4.097150817605763
- type: nauc_precision_at_20_diff1
value: 94.9723479135242
- type: nauc_precision_at_20_max
value: 91.04000574588684
- type: nauc_precision_at_20_std
value: 48.764634058749586
- type: nauc_precision_at_3_diff1
value: 90.52690041533852
- type: nauc_precision_at_3_max
value: 84.35075179497126
- type: nauc_precision_at_3_std
value: 12.036768730480507
- type: nauc_precision_at_5_diff1
value: 90.44234360410769
- type: nauc_precision_at_5_max
value: 83.21895424836558
- type: nauc_precision_at_5_std
value: 9.974323062558037
- type: nauc_recall_at_1000_diff1
value: .nan
- type: nauc_recall_at_1000_max
value: .nan
- type: nauc_recall_at_1000_std
value: .nan
- type: nauc_recall_at_100_diff1
value: 93.46405228758294
- type: nauc_recall_at_100_max
value: 100.0
- type: nauc_recall_at_100_std
value: 70.71661998132666
- type: nauc_recall_at_10_diff1
value: 90.13938908896864
- type: nauc_recall_at_10_max
value: 82.21121782046124
- type: nauc_recall_at_10_std
value: 13.075230092036506
- type: nauc_recall_at_1_diff1
value: 93.29828518358822
- type: nauc_recall_at_1_max
value: 78.69539619887887
- type: nauc_recall_at_1_std
value: -4.097150817605763
- type: nauc_recall_at_20_diff1
value: 94.97234791352489
- type: nauc_recall_at_20_max
value: 91.04000574588774
- type: nauc_recall_at_20_std
value: 48.764634058752065
- type: nauc_recall_at_3_diff1
value: 90.52690041533845
- type: nauc_recall_at_3_max
value: 84.35075179497079
- type: nauc_recall_at_3_std
value: 12.036768730480583
- type: nauc_recall_at_5_diff1
value: 90.44234360410861
- type: nauc_recall_at_5_max
value: 83.21895424836595
- type: nauc_recall_at_5_std
value: 9.974323062558147
- type: ndcg_at_1
value: 87.111
- type: ndcg_at_10
value: 92.23599999999999
- type: ndcg_at_100
value: 92.87100000000001
- type: ndcg_at_1000
value: 92.928
- type: ndcg_at_20
value: 92.67699999999999
- type: ndcg_at_3
value: 90.973
- type: ndcg_at_5
value: 91.801
- type: precision_at_1
value: 87.111
- type: precision_at_10
value: 9.689
- type: precision_at_100
value: 0.996
- type: precision_at_1000
value: 0.1
- type: precision_at_20
value: 4.928
- type: precision_at_3
value: 31.185000000000002
- type: precision_at_5
value: 19.111
- type: recall_at_1
value: 87.111
- type: recall_at_10
value: 96.88900000000001
- type: recall_at_100
value: 99.556
- type: recall_at_1000
value: 100.0
- type: recall_at_20
value: 98.556
- type: recall_at_3
value: 93.556
- type: recall_at_5
value: 95.556
task:
type: Retrieval
- dataset:
config: rus_Cyrl-eng_Latn
name: MTEB BelebeleRetrieval (rus_Cyrl-eng_Latn)
revision: 75b399394a9803252cfec289d103de462763db7c
split: test
type: facebook/belebele
metrics:
- type: main_score
value: 86.615
- type: map_at_1
value: 78.0
- type: map_at_10
value: 83.822
- type: map_at_100
value: 84.033
- type: map_at_1000
value: 84.03500000000001
- type: map_at_20
value: 83.967
- type: map_at_3
value: 82.315
- type: map_at_5
value: 83.337
- type: mrr_at_1
value: 78.0
- type: mrr_at_10
value: 83.82213403880073
- type: mrr_at_100
value: 84.03281327810801
- type: mrr_at_1000
value: 84.03460051000452
- type: mrr_at_20
value: 83.9673773122303
- type: mrr_at_3
value: 82.31481481481484
- type: mrr_at_5
value: 83.33703703703708
- type: nauc_map_at_1000_diff1
value: 80.78467576987832
- type: nauc_map_at_1000_max
value: 51.41718334647604
- type: nauc_map_at_1000_std
value: -16.23873782768812
- type: nauc_map_at_100_diff1
value: 80.78490931240695
- type: nauc_map_at_100_max
value: 51.41504597713061
- type: nauc_map_at_100_std
value: -16.23538559475366
- type: nauc_map_at_10_diff1
value: 80.73989245374868
- type: nauc_map_at_10_max
value: 51.43026079433827
- type: nauc_map_at_10_std
value: -16.13414330905897
- type: nauc_map_at_1_diff1
value: 82.36966971144186
- type: nauc_map_at_1_max
value: 52.988877039509916
- type: nauc_map_at_1_std
value: -15.145824639495546
- type: nauc_map_at_20_diff1
value: 80.75923781626145
- type: nauc_map_at_20_max
value: 51.40181079374639
- type: nauc_map_at_20_std
value: -16.260566097377165
- type: nauc_map_at_3_diff1
value: 80.65242627065471
- type: nauc_map_at_3_max
value: 50.623980338841214
- type: nauc_map_at_3_std
value: -16.818343442794294
- type: nauc_map_at_5_diff1
value: 80.45976387021862
- type: nauc_map_at_5_max
value: 51.533621728445866
- type: nauc_map_at_5_std
value: -16.279891536945815
- type: nauc_mrr_at_1000_diff1
value: 80.78467576987832
- type: nauc_mrr_at_1000_max
value: 51.41718334647604
- type: nauc_mrr_at_1000_std
value: -16.23873782768812
- type: nauc_mrr_at_100_diff1
value: 80.78490931240695
- type: nauc_mrr_at_100_max
value: 51.41504597713061
- type: nauc_mrr_at_100_std
value: -16.23538559475366
- type: nauc_mrr_at_10_diff1
value: 80.73989245374868
- type: nauc_mrr_at_10_max
value: 51.43026079433827
- type: nauc_mrr_at_10_std
value: -16.13414330905897
- type: nauc_mrr_at_1_diff1
value: 82.36966971144186
- type: nauc_mrr_at_1_max
value: 52.988877039509916
- type: nauc_mrr_at_1_std
value: -15.145824639495546
- type: nauc_mrr_at_20_diff1
value: 80.75923781626145
- type: nauc_mrr_at_20_max
value: 51.40181079374639
- type: nauc_mrr_at_20_std
value: -16.260566097377165
- type: nauc_mrr_at_3_diff1
value: 80.65242627065471
- type: nauc_mrr_at_3_max
value: 50.623980338841214
- type: nauc_mrr_at_3_std
value: -16.818343442794294
- type: nauc_mrr_at_5_diff1
value: 80.45976387021862
- type: nauc_mrr_at_5_max
value: 51.533621728445866
- type: nauc_mrr_at_5_std
value: -16.279891536945815
- type: nauc_ndcg_at_1000_diff1
value: 80.60009446938174
- type: nauc_ndcg_at_1000_max
value: 51.381708043594166
- type: nauc_ndcg_at_1000_std
value: -16.054256944160848
- type: nauc_ndcg_at_100_diff1
value: 80.58971462930421
- type: nauc_ndcg_at_100_max
value: 51.25436917735444
- type: nauc_ndcg_at_100_std
value: -15.862944972269894
- type: nauc_ndcg_at_10_diff1
value: 80.37967179454489
- type: nauc_ndcg_at_10_max
value: 51.590394257251006
- type: nauc_ndcg_at_10_std
value: -15.489799384799591
- type: nauc_ndcg_at_1_diff1
value: 82.36966971144186
- type: nauc_ndcg_at_1_max
value: 52.988877039509916
- type: nauc_ndcg_at_1_std
value: -15.145824639495546
- type: nauc_ndcg_at_20_diff1
value: 80.40299527470081
- type: nauc_ndcg_at_20_max
value: 51.395132284307074
- type: nauc_ndcg_at_20_std
value: -15.906165526937203
- type: nauc_ndcg_at_3_diff1
value: 80.10347913649302
- type: nauc_ndcg_at_3_max
value: 50.018431855573844
- type: nauc_ndcg_at_3_std
value: -17.12743750163884
- type: nauc_ndcg_at_5_diff1
value: 79.65918647776613
- type: nauc_ndcg_at_5_max
value: 51.76710880330806
- type: nauc_ndcg_at_5_std
value: -16.071901882035945
- type: nauc_precision_at_1000_diff1
value: .nan
- type: nauc_precision_at_1000_max
value: .nan
- type: nauc_precision_at_1000_std
value: .nan
- type: nauc_precision_at_100_diff1
value: 77.41596638655459
- type: nauc_precision_at_100_max
value: 22.572362278246565
- type: nauc_precision_at_100_std
value: 26.890756302525716
- type: nauc_precision_at_10_diff1
value: 77.82112845138009
- type: nauc_precision_at_10_max
value: 54.2550353474723
- type: nauc_precision_at_10_std
value: -7.492997198879646
- type: nauc_precision_at_1_diff1
value: 82.36966971144186
- type: nauc_precision_at_1_max
value: 52.988877039509916
- type: nauc_precision_at_1_std
value: -15.145824639495546
- type: nauc_precision_at_20_diff1
value: 75.89091192032318
- type: nauc_precision_at_20_max
value: 52.03275754746293
- type: nauc_precision_at_20_std
value: -7.8411920323686175
- type: nauc_precision_at_3_diff1
value: 78.0256020644638
- type: nauc_precision_at_3_max
value: 47.80353641248523
- type: nauc_precision_at_3_std
value: -18.181625255723503
- type: nauc_precision_at_5_diff1
value: 75.21583976056174
- type: nauc_precision_at_5_max
value: 53.716281032960765
- type: nauc_precision_at_5_std
value: -14.411700753360812
- type: nauc_recall_at_1000_diff1
value: .nan
- type: nauc_recall_at_1000_max
value: .nan
- type: nauc_recall_at_1000_std
value: .nan
- type: nauc_recall_at_100_diff1
value: 77.4159663865523
- type: nauc_recall_at_100_max
value: 22.57236227824646
- type: nauc_recall_at_100_std
value: 26.89075630252133
- type: nauc_recall_at_10_diff1
value: 77.82112845138037
- type: nauc_recall_at_10_max
value: 54.25503534747204
- type: nauc_recall_at_10_std
value: -7.492997198879666
- type: nauc_recall_at_1_diff1
value: 82.36966971144186
- type: nauc_recall_at_1_max
value: 52.988877039509916
- type: nauc_recall_at_1_std
value: -15.145824639495546
- type: nauc_recall_at_20_diff1
value: 75.89091192032362
- type: nauc_recall_at_20_max
value: 52.032757547463184
- type: nauc_recall_at_20_std
value: -7.84119203236888
- type: nauc_recall_at_3_diff1
value: 78.02560206446354
- type: nauc_recall_at_3_max
value: 47.80353641248526
- type: nauc_recall_at_3_std
value: -18.181625255723656
- type: nauc_recall_at_5_diff1
value: 75.21583976056185
- type: nauc_recall_at_5_max
value: 53.71628103296118
- type: nauc_recall_at_5_std
value: -14.411700753360634
- type: ndcg_at_1
value: 78.0
- type: ndcg_at_10
value: 86.615
- type: ndcg_at_100
value: 87.558
- type: ndcg_at_1000
value: 87.613
- type: ndcg_at_20
value: 87.128
- type: ndcg_at_3
value: 83.639
- type: ndcg_at_5
value: 85.475
- type: precision_at_1
value: 78.0
- type: precision_at_10
value: 9.533
- type: precision_at_100
value: 0.996
- type: precision_at_1000
value: 0.1
- type: precision_at_20
value: 4.867
- type: precision_at_3
value: 29.148000000000003
- type: precision_at_5
value: 18.378
- type: recall_at_1
value: 78.0
- type: recall_at_10
value: 95.333
- type: recall_at_100
value: 99.556
- type: recall_at_1000
value: 100.0
- type: recall_at_20
value: 97.333
- type: recall_at_3
value: 87.444
- type: recall_at_5
value: 91.889
task:
type: Retrieval
- dataset:
config: eng_Latn-rus_Cyrl
name: MTEB BelebeleRetrieval (eng_Latn-rus_Cyrl)
revision: 75b399394a9803252cfec289d103de462763db7c
split: test
type: facebook/belebele
metrics:
- type: main_score
value: 82.748
- type: map_at_1
value: 73.444
- type: map_at_10
value: 79.857
- type: map_at_100
value: 80.219
- type: map_at_1000
value: 80.22500000000001
- type: map_at_20
value: 80.10300000000001
- type: map_at_3
value: 78.593
- type: map_at_5
value: 79.515
- type: mrr_at_1
value: 73.44444444444444
- type: mrr_at_10
value: 79.85705467372136
- type: mrr_at_100
value: 80.21942320422542
- type: mrr_at_1000
value: 80.2245364027152
- type: mrr_at_20
value: 80.10273201266493
- type: mrr_at_3
value: 78.59259259259258
- type: mrr_at_5
value: 79.51481481481483
- type: nauc_map_at_1000_diff1
value: 83.69682652271125
- type: nauc_map_at_1000_max
value: 61.70131708044767
- type: nauc_map_at_1000_std
value: 9.345825405274955
- type: nauc_map_at_100_diff1
value: 83.68924820523492
- type: nauc_map_at_100_max
value: 61.6965735573098
- type: nauc_map_at_100_std
value: 9.366132859525775
- type: nauc_map_at_10_diff1
value: 83.61802964269985
- type: nauc_map_at_10_max
value: 61.74274476167882
- type: nauc_map_at_10_std
value: 9.504060995819101
- type: nauc_map_at_1_diff1
value: 86.37079221403225
- type: nauc_map_at_1_max
value: 61.856861655370686
- type: nauc_map_at_1_std
value: 4.708911881992707
- type: nauc_map_at_20_diff1
value: 83.62920965453047
- type: nauc_map_at_20_max
value: 61.761029350326965
- type: nauc_map_at_20_std
value: 9.572978651118351
- type: nauc_map_at_3_diff1
value: 83.66665673154306
- type: nauc_map_at_3_max
value: 61.13597610587937
- type: nauc_map_at_3_std
value: 9.309596395240598
- type: nauc_map_at_5_diff1
value: 83.52307226455358
- type: nauc_map_at_5_max
value: 61.59405758027573
- type: nauc_map_at_5_std
value: 9.320025423287671
- type: nauc_mrr_at_1000_diff1
value: 83.69682652271125
- type: nauc_mrr_at_1000_max
value: 61.70131708044767
- type: nauc_mrr_at_1000_std
value: 9.345825405274955
- type: nauc_mrr_at_100_diff1
value: 83.68924820523492
- type: nauc_mrr_at_100_max
value: 61.6965735573098
- type: nauc_mrr_at_100_std
value: 9.366132859525775
- type: nauc_mrr_at_10_diff1
value: 83.61802964269985
- type: nauc_mrr_at_10_max
value: 61.74274476167882
- type: nauc_mrr_at_10_std
value: 9.504060995819101
- type: nauc_mrr_at_1_diff1
value: 86.37079221403225
- type: nauc_mrr_at_1_max
value: 61.856861655370686
- type: nauc_mrr_at_1_std
value: 4.708911881992707
- type: nauc_mrr_at_20_diff1
value: 83.62920965453047
- type: nauc_mrr_at_20_max
value: 61.761029350326965
- type: nauc_mrr_at_20_std
value: 9.572978651118351
- type: nauc_mrr_at_3_diff1
value: 83.66665673154306
- type: nauc_mrr_at_3_max
value: 61.13597610587937
- type: nauc_mrr_at_3_std
value: 9.309596395240598
- type: nauc_mrr_at_5_diff1
value: 83.52307226455358
- type: nauc_mrr_at_5_max
value: 61.59405758027573
- type: nauc_mrr_at_5_std
value: 9.320025423287671
- type: nauc_ndcg_at_1000_diff1
value: 83.24213186482201
- type: nauc_ndcg_at_1000_max
value: 61.77629841787496
- type: nauc_ndcg_at_1000_std
value: 10.332527869705851
- type: nauc_ndcg_at_100_diff1
value: 83.06815820441027
- type: nauc_ndcg_at_100_max
value: 61.6947181864579
- type: nauc_ndcg_at_100_std
value: 10.888922975877316
- type: nauc_ndcg_at_10_diff1
value: 82.58238431386295
- type: nauc_ndcg_at_10_max
value: 62.10333663935709
- type: nauc_ndcg_at_10_std
value: 11.746030330958174
- type: nauc_ndcg_at_1_diff1
value: 86.37079221403225
- type: nauc_ndcg_at_1_max
value: 61.856861655370686
- type: nauc_ndcg_at_1_std
value: 4.708911881992707
- type: nauc_ndcg_at_20_diff1
value: 82.67888324480154
- type: nauc_ndcg_at_20_max
value: 62.28124917486516
- type: nauc_ndcg_at_20_std
value: 12.343058917563914
- type: nauc_ndcg_at_3_diff1
value: 82.71277373710663
- type: nauc_ndcg_at_3_max
value: 60.66677922989939
- type: nauc_ndcg_at_3_std
value: 10.843633736296528
- type: nauc_ndcg_at_5_diff1
value: 82.34691124846786
- type: nauc_ndcg_at_5_max
value: 61.605961382062716
- type: nauc_ndcg_at_5_std
value: 11.129011077702602
- type: nauc_precision_at_1000_diff1
value: .nan
- type: nauc_precision_at_1000_max
value: .nan
- type: nauc_precision_at_1000_std
value: .nan
- type: nauc_precision_at_100_diff1
value: 60.93103908230194
- type: nauc_precision_at_100_max
value: 52.621048419370695
- type: nauc_precision_at_100_std
value: 85.60090702947922
- type: nauc_precision_at_10_diff1
value: 76.26517273576093
- type: nauc_precision_at_10_max
value: 65.2013694366636
- type: nauc_precision_at_10_std
value: 26.50357920946173
- type: nauc_precision_at_1_diff1
value: 86.37079221403225
- type: nauc_precision_at_1_max
value: 61.856861655370686
- type: nauc_precision_at_1_std
value: 4.708911881992707
- type: nauc_precision_at_20_diff1
value: 73.47946930710295
- type: nauc_precision_at_20_max
value: 70.19520986689217
- type: nauc_precision_at_20_std
value: 45.93186111653967
- type: nauc_precision_at_3_diff1
value: 79.02026879450186
- type: nauc_precision_at_3_max
value: 58.75074624692399
- type: nauc_precision_at_3_std
value: 16.740684654251037
- type: nauc_precision_at_5_diff1
value: 76.47585662281637
- type: nauc_precision_at_5_max
value: 61.86270922013127
- type: nauc_precision_at_5_std
value: 20.1833625455035
- type: nauc_recall_at_1000_diff1
value: .nan
- type: nauc_recall_at_1000_max
value: .nan
- type: nauc_recall_at_1000_std
value: .nan
- type: nauc_recall_at_100_diff1
value: 60.93103908229921
- type: nauc_recall_at_100_max
value: 52.62104841936668
- type: nauc_recall_at_100_std
value: 85.60090702947748
- type: nauc_recall_at_10_diff1
value: 76.26517273576097
- type: nauc_recall_at_10_max
value: 65.20136943666347
- type: nauc_recall_at_10_std
value: 26.50357920946174
- type: nauc_recall_at_1_diff1
value: 86.37079221403225
- type: nauc_recall_at_1_max
value: 61.856861655370686
- type: nauc_recall_at_1_std
value: 4.708911881992707
- type: nauc_recall_at_20_diff1
value: 73.47946930710269
- type: nauc_recall_at_20_max
value: 70.19520986689254
- type: nauc_recall_at_20_std
value: 45.93186111653943
- type: nauc_recall_at_3_diff1
value: 79.02026879450173
- type: nauc_recall_at_3_max
value: 58.750746246923924
- type: nauc_recall_at_3_std
value: 16.740684654251076
- type: nauc_recall_at_5_diff1
value: 76.4758566228162
- type: nauc_recall_at_5_max
value: 61.862709220131386
- type: nauc_recall_at_5_std
value: 20.18336254550361
- type: ndcg_at_1
value: 73.444
- type: ndcg_at_10
value: 82.748
- type: ndcg_at_100
value: 84.416
- type: ndcg_at_1000
value: 84.52300000000001
- type: ndcg_at_20
value: 83.646
- type: ndcg_at_3
value: 80.267
- type: ndcg_at_5
value: 81.922
- type: precision_at_1
value: 73.444
- type: precision_at_10
value: 9.167
- type: precision_at_100
value: 0.992
- type: precision_at_1000
value: 0.1
- type: precision_at_20
value: 4.761
- type: precision_at_3
value: 28.37
- type: precision_at_5
value: 17.822
- type: recall_at_1
value: 73.444
- type: recall_at_10
value: 91.667
- type: recall_at_100
value: 99.222
- type: recall_at_1000
value: 100.0
- type: recall_at_20
value: 95.222
- type: recall_at_3
value: 85.111
- type: recall_at_5
value: 89.11099999999999
task:
type: Retrieval
- dataset:
config: eng_Latn-rus_Cyrl
name: MTEB BibleNLPBitextMining (eng_Latn-rus_Cyrl)
revision: 264a18480c529d9e922483839b4b9758e690b762
split: train
type: davidstap/biblenlp-corpus-mmteb
metrics:
- type: accuracy
value: 96.875
- type: f1
value: 95.83333333333333
- type: main_score
value: 95.83333333333333
- type: precision
value: 95.3125
- type: recall
value: 96.875
task:
type: BitextMining
- dataset:
config: rus_Cyrl-eng_Latn
name: MTEB BibleNLPBitextMining (rus_Cyrl-eng_Latn)
revision: 264a18480c529d9e922483839b4b9758e690b762
split: train
type: davidstap/biblenlp-corpus-mmteb
metrics:
- type: accuracy
value: 88.671875
- type: f1
value: 85.3515625
- type: main_score
value: 85.3515625
- type: precision
value: 83.85416666666667
- type: recall
value: 88.671875
task:
type: BitextMining
- dataset:
config: default
name: MTEB CEDRClassification (default)
revision: c0ba03d058e3e1b2f3fd20518875a4563dd12db4
split: test
type: ai-forever/cedr-classification
metrics:
- type: accuracy
value: 40.06907545164719
- type: f1
value: 26.285000550712407
- type: lrap
value: 64.4280021253997
- type: main_score
value: 40.06907545164719
task:
type: MultilabelClassification
- dataset:
config: default
name: MTEB CyrillicTurkicLangClassification (default)
revision: e42d330f33d65b7b72dfd408883daf1661f06f18
split: test
type: tatiana-merz/cyrillic_turkic_langs
metrics:
- type: accuracy
value: 43.3447265625
- type: f1
value: 40.08400146827895
- type: f1_weighted
value: 40.08499428040896
- type: main_score
value: 43.3447265625
task:
type: Classification
- dataset:
config: ace_Arab-rus_Cyrl
name: MTEB FloresBitextMining (ace_Arab-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 6.225296442687747
- type: f1
value: 5.5190958860075
- type: main_score
value: 5.5190958860075
- type: precision
value: 5.3752643758000005
- type: recall
value: 6.225296442687747
task:
type: BitextMining
- dataset:
config: bam_Latn-rus_Cyrl
name: MTEB FloresBitextMining (bam_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 68.37944664031622
- type: f1
value: 64.54819836666252
- type: main_score
value: 64.54819836666252
- type: precision
value: 63.07479233454916
- type: recall
value: 68.37944664031622
task:
type: BitextMining
- dataset:
config: dzo_Tibt-rus_Cyrl
name: MTEB FloresBitextMining (dzo_Tibt-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 0.09881422924901186
- type: f1
value: 0.00019509225912934226
- type: main_score
value: 0.00019509225912934226
- type: precision
value: 9.76425190207627e-05
- type: recall
value: 0.09881422924901186
task:
type: BitextMining
- dataset:
config: hin_Deva-rus_Cyrl
name: MTEB FloresBitextMining (hin_Deva-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.60474308300395
- type: f1
value: 99.47299077733861
- type: main_score
value: 99.47299077733861
- type: precision
value: 99.40711462450594
- type: recall
value: 99.60474308300395
task:
type: BitextMining
- dataset:
config: khm_Khmr-rus_Cyrl
name: MTEB FloresBitextMining (khm_Khmr-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 88.83399209486166
- type: f1
value: 87.71151056318254
- type: main_score
value: 87.71151056318254
- type: precision
value: 87.32012500709193
- type: recall
value: 88.83399209486166
task:
type: BitextMining
- dataset:
config: mag_Deva-rus_Cyrl
name: MTEB FloresBitextMining (mag_Deva-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.02371541501977
- type: f1
value: 97.7239789196311
- type: main_score
value: 97.7239789196311
- type: precision
value: 97.61904761904762
- type: recall
value: 98.02371541501977
task:
type: BitextMining
- dataset:
config: pap_Latn-rus_Cyrl
name: MTEB FloresBitextMining (pap_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 94.0711462450593
- type: f1
value: 93.68187806922984
- type: main_score
value: 93.68187806922984
- type: precision
value: 93.58925452707051
- type: recall
value: 94.0711462450593
task:
type: BitextMining
- dataset:
config: sot_Latn-rus_Cyrl
name: MTEB FloresBitextMining (sot_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 90.9090909090909
- type: f1
value: 89.23171936758892
- type: main_score
value: 89.23171936758892
- type: precision
value: 88.51790014083866
- type: recall
value: 90.9090909090909
task:
type: BitextMining
- dataset:
config: tur_Latn-rus_Cyrl
name: MTEB FloresBitextMining (tur_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.2094861660079
- type: f1
value: 98.9459815546772
- type: main_score
value: 98.9459815546772
- type: precision
value: 98.81422924901186
- type: recall
value: 99.2094861660079
task:
type: BitextMining
- dataset:
config: ace_Latn-rus_Cyrl
name: MTEB FloresBitextMining (ace_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 66.10671936758892
- type: f1
value: 63.81888256297873
- type: main_score
value: 63.81888256297873
- type: precision
value: 63.01614067933451
- type: recall
value: 66.10671936758892
task:
type: BitextMining
- dataset:
config: ban_Latn-rus_Cyrl
name: MTEB FloresBitextMining (ban_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 79.44664031620553
- type: f1
value: 77.6311962082713
- type: main_score
value: 77.6311962082713
- type: precision
value: 76.93977931929739
- type: recall
value: 79.44664031620553
task:
type: BitextMining
- dataset:
config: ell_Grek-rus_Cyrl
name: MTEB FloresBitextMining (ell_Grek-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.40711462450594
- type: f1
value: 99.2094861660079
- type: main_score
value: 99.2094861660079
- type: precision
value: 99.1106719367589
- type: recall
value: 99.40711462450594
task:
type: BitextMining
- dataset:
config: hne_Deva-rus_Cyrl
name: MTEB FloresBitextMining (hne_Deva-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 96.83794466403161
- type: f1
value: 96.25352907961603
- type: main_score
value: 96.25352907961603
- type: precision
value: 96.02155091285526
- type: recall
value: 96.83794466403161
task:
type: BitextMining
- dataset:
config: kik_Latn-rus_Cyrl
name: MTEB FloresBitextMining (kik_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 76.28458498023716
- type: f1
value: 73.5596919895859
- type: main_score
value: 73.5596919895859
- type: precision
value: 72.40900759055246
- type: recall
value: 76.28458498023716
task:
type: BitextMining
- dataset:
config: mai_Deva-rus_Cyrl
name: MTEB FloresBitextMining (mai_Deva-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.72727272727273
- type: f1
value: 97.37812911725956
- type: main_score
value: 97.37812911725956
- type: precision
value: 97.26002258610953
- type: recall
value: 97.72727272727273
task:
type: BitextMining
- dataset:
config: pbt_Arab-rus_Cyrl
name: MTEB FloresBitextMining (pbt_Arab-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 94.0711462450593
- type: f1
value: 93.34700387331966
- type: main_score
value: 93.34700387331966
- type: precision
value: 93.06920556920556
- type: recall
value: 94.0711462450593
task:
type: BitextMining
- dataset:
config: spa_Latn-rus_Cyrl
name: MTEB FloresBitextMining (spa_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.2094861660079
- type: f1
value: 98.9459815546772
- type: main_score
value: 98.9459815546772
- type: precision
value: 98.81422924901186
- type: recall
value: 99.2094861660079
task:
type: BitextMining
- dataset:
config: twi_Latn-rus_Cyrl
name: MTEB FloresBitextMining (twi_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 80.73122529644269
- type: f1
value: 77.77434363246721
- type: main_score
value: 77.77434363246721
- type: precision
value: 76.54444287596462
- type: recall
value: 80.73122529644269
task:
type: BitextMining
- dataset:
config: acm_Arab-rus_Cyrl
name: MTEB FloresBitextMining (acm_Arab-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 94.56521739130434
- type: f1
value: 92.92490118577075
- type: main_score
value: 92.92490118577075
- type: precision
value: 92.16897233201581
- type: recall
value: 94.56521739130434
task:
type: BitextMining
- dataset:
config: bel_Cyrl-rus_Cyrl
name: MTEB FloresBitextMining (bel_Cyrl-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.2094861660079
- type: f1
value: 98.98550724637681
- type: main_score
value: 98.98550724637681
- type: precision
value: 98.88833992094862
- type: recall
value: 99.2094861660079
task:
type: BitextMining
- dataset:
config: eng_Latn-rus_Cyrl
name: MTEB FloresBitextMining (eng_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.60474308300395
- type: f1
value: 99.4729907773386
- type: main_score
value: 99.4729907773386
- type: precision
value: 99.40711462450594
- type: recall
value: 99.60474308300395
task:
type: BitextMining
- dataset:
config: hrv_Latn-rus_Cyrl
name: MTEB FloresBitextMining (hrv_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.2094861660079
- type: f1
value: 99.05138339920948
- type: main_score
value: 99.05138339920948
- type: precision
value: 99.00691699604744
- type: recall
value: 99.2094861660079
task:
type: BitextMining
- dataset:
config: kin_Latn-rus_Cyrl
name: MTEB FloresBitextMining (kin_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 88.2411067193676
- type: f1
value: 86.5485246227658
- type: main_score
value: 86.5485246227658
- type: precision
value: 85.90652101521667
- type: recall
value: 88.2411067193676
task:
type: BitextMining
- dataset:
config: mal_Mlym-rus_Cyrl
name: MTEB FloresBitextMining (mal_Mlym-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.51778656126481
- type: f1
value: 98.07971014492753
- type: main_score
value: 98.07971014492753
- type: precision
value: 97.88372859025033
- type: recall
value: 98.51778656126481
task:
type: BitextMining
- dataset:
config: pes_Arab-rus_Cyrl
name: MTEB FloresBitextMining (pes_Arab-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.51778656126481
- type: f1
value: 98.0566534914361
- type: main_score
value: 98.0566534914361
- type: precision
value: 97.82608695652173
- type: recall
value: 98.51778656126481
task:
type: BitextMining
- dataset:
config: srd_Latn-rus_Cyrl
name: MTEB FloresBitextMining (srd_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 82.6086956521739
- type: f1
value: 80.9173470979821
- type: main_score
value: 80.9173470979821
- type: precision
value: 80.24468672882627
- type: recall
value: 82.6086956521739
task:
type: BitextMining
- dataset:
config: tzm_Tfng-rus_Cyrl
name: MTEB FloresBitextMining (tzm_Tfng-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 7.41106719367589
- type: f1
value: 6.363562740945329
- type: main_score
value: 6.363562740945329
- type: precision
value: 6.090373175353411
- type: recall
value: 7.41106719367589
task:
type: BitextMining
- dataset:
config: acq_Arab-rus_Cyrl
name: MTEB FloresBitextMining (acq_Arab-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 95.25691699604744
- type: f1
value: 93.81422924901187
- type: main_score
value: 93.81422924901187
- type: precision
value: 93.14064558629775
- type: recall
value: 95.25691699604744
task:
type: BitextMining
- dataset:
config: bem_Latn-rus_Cyrl
name: MTEB FloresBitextMining (bem_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 68.08300395256917
- type: f1
value: 65.01368772860867
- type: main_score
value: 65.01368772860867
- type: precision
value: 63.91052337510628
- type: recall
value: 68.08300395256917
task:
type: BitextMining
- dataset:
config: epo_Latn-rus_Cyrl
name: MTEB FloresBitextMining (epo_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.41897233201581
- type: f1
value: 98.17193675889328
- type: main_score
value: 98.17193675889328
- type: precision
value: 98.08210564139418
- type: recall
value: 98.41897233201581
task:
type: BitextMining
- dataset:
config: hun_Latn-rus_Cyrl
name: MTEB FloresBitextMining (hun_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.30830039525692
- type: f1
value: 99.1106719367589
- type: main_score
value: 99.1106719367589
- type: precision
value: 99.01185770750988
- type: recall
value: 99.30830039525692
task:
type: BitextMining
- dataset:
config: kir_Cyrl-rus_Cyrl
name: MTEB FloresBitextMining (kir_Cyrl-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.5296442687747
- type: f1
value: 97.07549806364035
- type: main_score
value: 97.07549806364035
- type: precision
value: 96.90958498023716
- type: recall
value: 97.5296442687747
task:
type: BitextMining
- dataset:
config: mar_Deva-rus_Cyrl
name: MTEB FloresBitextMining (mar_Deva-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.82608695652173
- type: f1
value: 97.44400527009222
- type: main_score
value: 97.44400527009222
- type: precision
value: 97.28966685488425
- type: recall
value: 97.82608695652173
task:
type: BitextMining
- dataset:
config: plt_Latn-rus_Cyrl
name: MTEB FloresBitextMining (plt_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 79.9407114624506
- type: f1
value: 78.3154177760691
- type: main_score
value: 78.3154177760691
- type: precision
value: 77.69877344877344
- type: recall
value: 79.9407114624506
task:
type: BitextMining
- dataset:
config: srp_Cyrl-rus_Cyrl
name: MTEB FloresBitextMining (srp_Cyrl-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.70355731225297
- type: f1
value: 99.60474308300395
- type: main_score
value: 99.60474308300395
- type: precision
value: 99.55533596837944
- type: recall
value: 99.70355731225297
task:
type: BitextMining
- dataset:
config: uig_Arab-rus_Cyrl
name: MTEB FloresBitextMining (uig_Arab-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 83.20158102766798
- type: f1
value: 81.44381923034585
- type: main_score
value: 81.44381923034585
- type: precision
value: 80.78813411582477
- type: recall
value: 83.20158102766798
task:
type: BitextMining
- dataset:
config: aeb_Arab-rus_Cyrl
name: MTEB FloresBitextMining (aeb_Arab-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 91.20553359683794
- type: f1
value: 88.75352907961603
- type: main_score
value: 88.75352907961603
- type: precision
value: 87.64328063241106
- type: recall
value: 91.20553359683794
task:
type: BitextMining
- dataset:
config: ben_Beng-rus_Cyrl
name: MTEB FloresBitextMining (ben_Beng-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.91304347826086
- type: f1
value: 98.60671936758894
- type: main_score
value: 98.60671936758894
- type: precision
value: 98.4766139657444
- type: recall
value: 98.91304347826086
task:
type: BitextMining
- dataset:
config: est_Latn-rus_Cyrl
name: MTEB FloresBitextMining (est_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 96.24505928853755
- type: f1
value: 95.27417027417027
- type: main_score
value: 95.27417027417027
- type: precision
value: 94.84107378129117
- type: recall
value: 96.24505928853755
task:
type: BitextMining
- dataset:
config: hye_Armn-rus_Cyrl
name: MTEB FloresBitextMining (hye_Armn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.02371541501977
- type: f1
value: 97.67786561264822
- type: main_score
value: 97.67786561264822
- type: precision
value: 97.55839022637441
- type: recall
value: 98.02371541501977
task:
type: BitextMining
- dataset:
config: kmb_Latn-rus_Cyrl
name: MTEB FloresBitextMining (kmb_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 46.047430830039524
- type: f1
value: 42.94464804804471
- type: main_score
value: 42.94464804804471
- type: precision
value: 41.9851895607238
- type: recall
value: 46.047430830039524
task:
type: BitextMining
- dataset:
config: min_Arab-rus_Cyrl
name: MTEB FloresBitextMining (min_Arab-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 3.9525691699604746
- type: f1
value: 3.402665192725756
- type: main_score
value: 3.402665192725756
- type: precision
value: 3.303787557740127
- type: recall
value: 3.9525691699604746
task:
type: BitextMining
- dataset:
config: pol_Latn-rus_Cyrl
name: MTEB FloresBitextMining (pol_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.60474308300395
- type: f1
value: 99.4729907773386
- type: main_score
value: 99.4729907773386
- type: precision
value: 99.40711462450594
- type: recall
value: 99.60474308300395
task:
type: BitextMining
- dataset:
config: ssw_Latn-rus_Cyrl
name: MTEB FloresBitextMining (ssw_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 73.22134387351778
- type: f1
value: 70.43086049508975
- type: main_score
value: 70.43086049508975
- type: precision
value: 69.35312022355656
- type: recall
value: 73.22134387351778
task:
type: BitextMining
- dataset:
config: ukr_Cyrl-rus_Cyrl
name: MTEB FloresBitextMining (ukr_Cyrl-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.90118577075098
- type: f1
value: 99.86824769433464
- type: main_score
value: 99.86824769433464
- type: precision
value: 99.85177865612648
- type: recall
value: 99.90118577075098
task:
type: BitextMining
- dataset:
config: afr_Latn-rus_Cyrl
name: MTEB FloresBitextMining (afr_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.2094861660079
- type: f1
value: 98.9459815546772
- type: main_score
value: 98.9459815546772
- type: precision
value: 98.81422924901186
- type: recall
value: 99.2094861660079
task:
type: BitextMining
- dataset:
config: bho_Deva-rus_Cyrl
name: MTEB FloresBitextMining (bho_Deva-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 94.0711462450593
- type: f1
value: 93.12182382834557
- type: main_score
value: 93.12182382834557
- type: precision
value: 92.7523453232338
- type: recall
value: 94.0711462450593
task:
type: BitextMining
- dataset:
config: eus_Latn-rus_Cyrl
name: MTEB FloresBitextMining (eus_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 92.19367588932806
- type: f1
value: 91.23604975587072
- type: main_score
value: 91.23604975587072
- type: precision
value: 90.86697443588663
- type: recall
value: 92.19367588932806
task:
type: BitextMining
- dataset:
config: ibo_Latn-rus_Cyrl
name: MTEB FloresBitextMining (ibo_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 82.21343873517787
- type: f1
value: 80.17901604858126
- type: main_score
value: 80.17901604858126
- type: precision
value: 79.3792284780028
- type: recall
value: 82.21343873517787
task:
type: BitextMining
- dataset:
config: kmr_Latn-rus_Cyrl
name: MTEB FloresBitextMining (kmr_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 68.67588932806325
- type: f1
value: 66.72311714750278
- type: main_score
value: 66.72311714750278
- type: precision
value: 66.00178401554004
- type: recall
value: 68.67588932806325
task:
type: BitextMining
- dataset:
config: min_Latn-rus_Cyrl
name: MTEB FloresBitextMining (min_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 78.65612648221344
- type: f1
value: 76.26592719972166
- type: main_score
value: 76.26592719972166
- type: precision
value: 75.39980459997484
- type: recall
value: 78.65612648221344
task:
type: BitextMining
- dataset:
config: por_Latn-rus_Cyrl
name: MTEB FloresBitextMining (por_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 96.83794466403161
- type: f1
value: 95.9669678147939
- type: main_score
value: 95.9669678147939
- type: precision
value: 95.59453227931488
- type: recall
value: 96.83794466403161
task:
type: BitextMining
- dataset:
config: sun_Latn-rus_Cyrl
name: MTEB FloresBitextMining (sun_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 92.4901185770751
- type: f1
value: 91.66553983773662
- type: main_score
value: 91.66553983773662
- type: precision
value: 91.34530928009188
- type: recall
value: 92.4901185770751
task:
type: BitextMining
- dataset:
config: umb_Latn-rus_Cyrl
name: MTEB FloresBitextMining (umb_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 41.00790513833992
- type: f1
value: 38.21319326004483
- type: main_score
value: 38.21319326004483
- type: precision
value: 37.200655467675546
- type: recall
value: 41.00790513833992
task:
type: BitextMining
- dataset:
config: ajp_Arab-rus_Cyrl
name: MTEB FloresBitextMining (ajp_Arab-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 95.35573122529645
- type: f1
value: 93.97233201581028
- type: main_score
value: 93.97233201581028
- type: precision
value: 93.33333333333333
- type: recall
value: 95.35573122529645
task:
type: BitextMining
- dataset:
config: bjn_Arab-rus_Cyrl
name: MTEB FloresBitextMining (bjn_Arab-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 3.6561264822134385
- type: f1
value: 3.1071978056336484
- type: main_score
value: 3.1071978056336484
- type: precision
value: 3.0039741229718215
- type: recall
value: 3.6561264822134385
task:
type: BitextMining
- dataset:
config: ewe_Latn-rus_Cyrl
name: MTEB FloresBitextMining (ewe_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 62.845849802371546
- type: f1
value: 59.82201175670472
- type: main_score
value: 59.82201175670472
- type: precision
value: 58.72629236362003
- type: recall
value: 62.845849802371546
task:
type: BitextMining
- dataset:
config: ilo_Latn-rus_Cyrl
name: MTEB FloresBitextMining (ilo_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 83.10276679841897
- type: f1
value: 80.75065288987582
- type: main_score
value: 80.75065288987582
- type: precision
value: 79.80726451662179
- type: recall
value: 83.10276679841897
task:
type: BitextMining
- dataset:
config: knc_Arab-rus_Cyrl
name: MTEB FloresBitextMining (knc_Arab-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 10.079051383399209
- type: f1
value: 8.759282456080921
- type: main_score
value: 8.759282456080921
- type: precision
value: 8.474735138956142
- type: recall
value: 10.079051383399209
task:
type: BitextMining
- dataset:
config: mkd_Cyrl-rus_Cyrl
name: MTEB FloresBitextMining (mkd_Cyrl-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.91304347826086
- type: f1
value: 98.55072463768116
- type: main_score
value: 98.55072463768116
- type: precision
value: 98.36956521739131
- type: recall
value: 98.91304347826086
task:
type: BitextMining
- dataset:
config: prs_Arab-rus_Cyrl
name: MTEB FloresBitextMining (prs_Arab-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.01185770750988
- type: f1
value: 98.68247694334651
- type: main_score
value: 98.68247694334651
- type: precision
value: 98.51778656126481
- type: recall
value: 99.01185770750988
task:
type: BitextMining
- dataset:
config: swe_Latn-rus_Cyrl
name: MTEB FloresBitextMining (swe_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.40711462450594
- type: f1
value: 99.22595520421606
- type: main_score
value: 99.22595520421606
- type: precision
value: 99.14361001317523
- type: recall
value: 99.40711462450594
task:
type: BitextMining
- dataset:
config: urd_Arab-rus_Cyrl
name: MTEB FloresBitextMining (urd_Arab-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.82608695652173
- type: f1
value: 97.25625823451911
- type: main_score
value: 97.25625823451911
- type: precision
value: 97.03063241106719
- type: recall
value: 97.82608695652173
task:
type: BitextMining
- dataset:
config: aka_Latn-rus_Cyrl
name: MTEB FloresBitextMining (aka_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 81.22529644268775
- type: f1
value: 77.94307687941227
- type: main_score
value: 77.94307687941227
- type: precision
value: 76.58782793293665
- type: recall
value: 81.22529644268775
task:
type: BitextMining
- dataset:
config: bjn_Latn-rus_Cyrl
name: MTEB FloresBitextMining (bjn_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 85.27667984189723
- type: f1
value: 83.6869192829922
- type: main_score
value: 83.6869192829922
- type: precision
value: 83.08670670691656
- type: recall
value: 85.27667984189723
task:
type: BitextMining
- dataset:
config: fao_Latn-rus_Cyrl
name: MTEB FloresBitextMining (fao_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 80.9288537549407
- type: f1
value: 79.29806087454745
- type: main_score
value: 79.29806087454745
- type: precision
value: 78.71445871526987
- type: recall
value: 80.9288537549407
task:
type: BitextMining
- dataset:
config: ind_Latn-rus_Cyrl
name: MTEB FloresBitextMining (ind_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.12252964426878
- type: f1
value: 97.5296442687747
- type: main_score
value: 97.5296442687747
- type: precision
value: 97.23320158102767
- type: recall
value: 98.12252964426878
task:
type: BitextMining
- dataset:
config: knc_Latn-rus_Cyrl
name: MTEB FloresBitextMining (knc_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 33.49802371541502
- type: f1
value: 32.02378215033989
- type: main_score
value: 32.02378215033989
- type: precision
value: 31.511356103747406
- type: recall
value: 33.49802371541502
task:
type: BitextMining
- dataset:
config: mlt_Latn-rus_Cyrl
name: MTEB FloresBitextMining (mlt_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 91.40316205533597
- type: f1
value: 90.35317684386006
- type: main_score
value: 90.35317684386006
- type: precision
value: 89.94845939633488
- type: recall
value: 91.40316205533597
task:
type: BitextMining
- dataset:
config: quy_Latn-rus_Cyrl
name: MTEB FloresBitextMining (quy_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 40.612648221343875
- type: f1
value: 38.74337544712602
- type: main_score
value: 38.74337544712602
- type: precision
value: 38.133716022178575
- type: recall
value: 40.612648221343875
task:
type: BitextMining
- dataset:
config: swh_Latn-rus_Cyrl
name: MTEB FloresBitextMining (swh_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.13438735177866
- type: f1
value: 96.47435897435898
- type: main_score
value: 96.47435897435898
- type: precision
value: 96.18741765480895
- type: recall
value: 97.13438735177866
task:
type: BitextMining
- dataset:
config: uzn_Latn-rus_Cyrl
name: MTEB FloresBitextMining (uzn_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 96.83794466403161
- type: f1
value: 96.26355528529442
- type: main_score
value: 96.26355528529442
- type: precision
value: 96.0501756697409
- type: recall
value: 96.83794466403161
task:
type: BitextMining
- dataset:
config: als_Latn-rus_Cyrl
name: MTEB FloresBitextMining (als_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.91304347826086
- type: f1
value: 98.6907114624506
- type: main_score
value: 98.6907114624506
- type: precision
value: 98.6142480707698
- type: recall
value: 98.91304347826086
task:
type: BitextMining
- dataset:
config: bod_Tibt-rus_Cyrl
name: MTEB FloresBitextMining (bod_Tibt-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 1.0869565217391304
- type: f1
value: 0.9224649610442628
- type: main_score
value: 0.9224649610442628
- type: precision
value: 0.8894275740459898
- type: recall
value: 1.0869565217391304
task:
type: BitextMining
- dataset:
config: fij_Latn-rus_Cyrl
name: MTEB FloresBitextMining (fij_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 63.24110671936759
- type: f1
value: 60.373189068189525
- type: main_score
value: 60.373189068189525
- type: precision
value: 59.32326368115546
- type: recall
value: 63.24110671936759
task:
type: BitextMining
- dataset:
config: isl_Latn-rus_Cyrl
name: MTEB FloresBitextMining (isl_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 89.03162055335969
- type: f1
value: 87.3102634715907
- type: main_score
value: 87.3102634715907
- type: precision
value: 86.65991814698712
- type: recall
value: 89.03162055335969
task:
type: BitextMining
- dataset:
config: kon_Latn-rus_Cyrl
name: MTEB FloresBitextMining (kon_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 73.91304347826086
- type: f1
value: 71.518235523573
- type: main_score
value: 71.518235523573
- type: precision
value: 70.58714102449801
- type: recall
value: 73.91304347826086
task:
type: BitextMining
- dataset:
config: mni_Beng-rus_Cyrl
name: MTEB FloresBitextMining (mni_Beng-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 29.545454545454547
- type: f1
value: 27.59513619889114
- type: main_score
value: 27.59513619889114
- type: precision
value: 26.983849851025344
- type: recall
value: 29.545454545454547
task:
type: BitextMining
- dataset:
config: ron_Latn-rus_Cyrl
name: MTEB FloresBitextMining (ron_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.40711462450594
- type: f1
value: 99.2094861660079
- type: main_score
value: 99.2094861660079
- type: precision
value: 99.1106719367589
- type: recall
value: 99.40711462450594
task:
type: BitextMining
- dataset:
config: szl_Latn-rus_Cyrl
name: MTEB FloresBitextMining (szl_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 86.26482213438736
- type: f1
value: 85.18912031587512
- type: main_score
value: 85.18912031587512
- type: precision
value: 84.77199409959775
- type: recall
value: 86.26482213438736
task:
type: BitextMining
- dataset:
config: vec_Latn-rus_Cyrl
name: MTEB FloresBitextMining (vec_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 85.67193675889328
- type: f1
value: 84.62529734716581
- type: main_score
value: 84.62529734716581
- type: precision
value: 84.2611422440705
- type: recall
value: 85.67193675889328
task:
type: BitextMining
- dataset:
config: amh_Ethi-rus_Cyrl
name: MTEB FloresBitextMining (amh_Ethi-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 94.76284584980237
- type: f1
value: 93.91735076517685
- type: main_score
value: 93.91735076517685
- type: precision
value: 93.57553798858147
- type: recall
value: 94.76284584980237
task:
type: BitextMining
- dataset:
config: bos_Latn-rus_Cyrl
name: MTEB FloresBitextMining (bos_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.2094861660079
- type: f1
value: 99.05655938264634
- type: main_score
value: 99.05655938264634
- type: precision
value: 99.01185770750988
- type: recall
value: 99.2094861660079
task:
type: BitextMining
- dataset:
config: fin_Latn-rus_Cyrl
name: MTEB FloresBitextMining (fin_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.02371541501977
- type: f1
value: 97.43741765480895
- type: main_score
value: 97.43741765480895
- type: precision
value: 97.1590909090909
- type: recall
value: 98.02371541501977
task:
type: BitextMining
- dataset:
config: ita_Latn-rus_Cyrl
name: MTEB FloresBitextMining (ita_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.70355731225297
- type: f1
value: 99.60474308300395
- type: main_score
value: 99.60474308300395
- type: precision
value: 99.55533596837944
- type: recall
value: 99.70355731225297
task:
type: BitextMining
- dataset:
config: kor_Hang-rus_Cyrl
name: MTEB FloresBitextMining (kor_Hang-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.33201581027669
- type: f1
value: 96.49868247694334
- type: main_score
value: 96.49868247694334
- type: precision
value: 96.10507246376811
- type: recall
value: 97.33201581027669
task:
type: BitextMining
- dataset:
config: mos_Latn-rus_Cyrl
name: MTEB FloresBitextMining (mos_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 34.683794466403164
- type: f1
value: 32.766819308009076
- type: main_score
value: 32.766819308009076
- type: precision
value: 32.1637493670237
- type: recall
value: 34.683794466403164
task:
type: BitextMining
- dataset:
config: run_Latn-rus_Cyrl
name: MTEB FloresBitextMining (run_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 83.399209486166
- type: f1
value: 81.10578750604326
- type: main_score
value: 81.10578750604326
- type: precision
value: 80.16763162673529
- type: recall
value: 83.399209486166
task:
type: BitextMining
- dataset:
config: tam_Taml-rus_Cyrl
name: MTEB FloresBitextMining (tam_Taml-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.41897233201581
- type: f1
value: 98.01548089591567
- type: main_score
value: 98.01548089591567
- type: precision
value: 97.84020327498588
- type: recall
value: 98.41897233201581
task:
type: BitextMining
- dataset:
config: vie_Latn-rus_Cyrl
name: MTEB FloresBitextMining (vie_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.1106719367589
- type: f1
value: 98.81422924901186
- type: main_score
value: 98.81422924901186
- type: precision
value: 98.66600790513834
- type: recall
value: 99.1106719367589
task:
type: BitextMining
- dataset:
config: apc_Arab-rus_Cyrl
name: MTEB FloresBitextMining (apc_Arab-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 93.87351778656127
- type: f1
value: 92.10803689064558
- type: main_score
value: 92.10803689064558
- type: precision
value: 91.30434782608695
- type: recall
value: 93.87351778656127
task:
type: BitextMining
- dataset:
config: bug_Latn-rus_Cyrl
name: MTEB FloresBitextMining (bug_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 57.608695652173914
- type: f1
value: 54.95878654927162
- type: main_score
value: 54.95878654927162
- type: precision
value: 54.067987427805654
- type: recall
value: 57.608695652173914
task:
type: BitextMining
- dataset:
config: fon_Latn-rus_Cyrl
name: MTEB FloresBitextMining (fon_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 61.95652173913043
- type: f1
value: 58.06537275812945
- type: main_score
value: 58.06537275812945
- type: precision
value: 56.554057596959204
- type: recall
value: 61.95652173913043
task:
type: BitextMining
- dataset:
config: jav_Latn-rus_Cyrl
name: MTEB FloresBitextMining (jav_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 93.47826086956522
- type: f1
value: 92.4784405318002
- type: main_score
value: 92.4784405318002
- type: precision
value: 92.09168143201127
- type: recall
value: 93.47826086956522
task:
type: BitextMining
- dataset:
config: lao_Laoo-rus_Cyrl
name: MTEB FloresBitextMining (lao_Laoo-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 91.10671936758892
- type: f1
value: 89.76104922745239
- type: main_score
value: 89.76104922745239
- type: precision
value: 89.24754593232855
- type: recall
value: 91.10671936758892
task:
type: BitextMining
- dataset:
config: mri_Latn-rus_Cyrl
name: MTEB FloresBitextMining (mri_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 71.14624505928853
- type: f1
value: 68.26947125119062
- type: main_score
value: 68.26947125119062
- type: precision
value: 67.15942311051006
- type: recall
value: 71.14624505928853
task:
type: BitextMining
- dataset:
config: rus_Cyrl-ace_Arab
name: MTEB FloresBitextMining (rus_Cyrl-ace_Arab)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 19.565217391304348
- type: f1
value: 16.321465000323805
- type: main_score
value: 16.321465000323805
- type: precision
value: 15.478527409347508
- type: recall
value: 19.565217391304348
task:
type: BitextMining
- dataset:
config: rus_Cyrl-bam_Latn
name: MTEB FloresBitextMining (rus_Cyrl-bam_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 73.41897233201581
- type: f1
value: 68.77366228182746
- type: main_score
value: 68.77366228182746
- type: precision
value: 66.96012924273795
- type: recall
value: 73.41897233201581
task:
type: BitextMining
- dataset:
config: rus_Cyrl-dzo_Tibt
name: MTEB FloresBitextMining (rus_Cyrl-dzo_Tibt)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 0.592885375494071
- type: f1
value: 0.02458062426370458
- type: main_score
value: 0.02458062426370458
- type: precision
value: 0.012824114724683876
- type: recall
value: 0.592885375494071
task:
type: BitextMining
- dataset:
config: rus_Cyrl-hin_Deva
name: MTEB FloresBitextMining (rus_Cyrl-hin_Deva)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.90118577075098
- type: f1
value: 99.86824769433464
- type: main_score
value: 99.86824769433464
- type: precision
value: 99.85177865612648
- type: recall
value: 99.90118577075098
task:
type: BitextMining
- dataset:
config: rus_Cyrl-khm_Khmr
name: MTEB FloresBitextMining (rus_Cyrl-khm_Khmr)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.13438735177866
- type: f1
value: 96.24505928853755
- type: main_score
value: 96.24505928853755
- type: precision
value: 95.81686429512516
- type: recall
value: 97.13438735177866
task:
type: BitextMining
- dataset:
config: rus_Cyrl-mag_Deva
name: MTEB FloresBitextMining (rus_Cyrl-mag_Deva)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.50592885375494
- type: f1
value: 99.35770750988142
- type: main_score
value: 99.35770750988142
- type: precision
value: 99.29183135704875
- type: recall
value: 99.50592885375494
task:
type: BitextMining
- dataset:
config: rus_Cyrl-pap_Latn
name: MTEB FloresBitextMining (rus_Cyrl-pap_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 96.93675889328063
- type: f1
value: 96.05072463768116
- type: main_score
value: 96.05072463768116
- type: precision
value: 95.66040843214758
- type: recall
value: 96.93675889328063
task:
type: BitextMining
- dataset:
config: rus_Cyrl-sot_Latn
name: MTEB FloresBitextMining (rus_Cyrl-sot_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 93.67588932806325
- type: f1
value: 91.7786561264822
- type: main_score
value: 91.7786561264822
- type: precision
value: 90.91238471673255
- type: recall
value: 93.67588932806325
task:
type: BitextMining
- dataset:
config: rus_Cyrl-tur_Latn
name: MTEB FloresBitextMining (rus_Cyrl-tur_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.01185770750988
- type: f1
value: 98.68247694334651
- type: main_score
value: 98.68247694334651
- type: precision
value: 98.51778656126481
- type: recall
value: 99.01185770750988
task:
type: BitextMining
- dataset:
config: rus_Cyrl-ace_Latn
name: MTEB FloresBitextMining (rus_Cyrl-ace_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 74.1106719367589
- type: f1
value: 70.21737923911836
- type: main_score
value: 70.21737923911836
- type: precision
value: 68.7068791410511
- type: recall
value: 74.1106719367589
task:
type: BitextMining
- dataset:
config: rus_Cyrl-ban_Latn
name: MTEB FloresBitextMining (rus_Cyrl-ban_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 81.7193675889328
- type: f1
value: 78.76470334510617
- type: main_score
value: 78.76470334510617
- type: precision
value: 77.76208475761422
- type: recall
value: 81.7193675889328
task:
type: BitextMining
- dataset:
config: rus_Cyrl-ell_Grek
name: MTEB FloresBitextMining (rus_Cyrl-ell_Grek)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.3201581027668
- type: f1
value: 97.76021080368908
- type: main_score
value: 97.76021080368908
- type: precision
value: 97.48023715415019
- type: recall
value: 98.3201581027668
task:
type: BitextMining
- dataset:
config: rus_Cyrl-hne_Deva
name: MTEB FloresBitextMining (rus_Cyrl-hne_Deva)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.51778656126481
- type: f1
value: 98.0566534914361
- type: main_score
value: 98.0566534914361
- type: precision
value: 97.82608695652173
- type: recall
value: 98.51778656126481
task:
type: BitextMining
- dataset:
config: rus_Cyrl-kik_Latn
name: MTEB FloresBitextMining (rus_Cyrl-kik_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 80.73122529644269
- type: f1
value: 76.42689244220864
- type: main_score
value: 76.42689244220864
- type: precision
value: 74.63877909530083
- type: recall
value: 80.73122529644269
task:
type: BitextMining
- dataset:
config: rus_Cyrl-mai_Deva
name: MTEB FloresBitextMining (rus_Cyrl-mai_Deva)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.91304347826086
- type: f1
value: 98.56719367588933
- type: main_score
value: 98.56719367588933
- type: precision
value: 98.40250329380763
- type: recall
value: 98.91304347826086
task:
type: BitextMining
- dataset:
config: rus_Cyrl-pbt_Arab
name: MTEB FloresBitextMining (rus_Cyrl-pbt_Arab)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.5296442687747
- type: f1
value: 96.73913043478261
- type: main_score
value: 96.73913043478261
- type: precision
value: 96.36034255599473
- type: recall
value: 97.5296442687747
task:
type: BitextMining
- dataset:
config: rus_Cyrl-spa_Latn
name: MTEB FloresBitextMining (rus_Cyrl-spa_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.40711462450594
- type: f1
value: 99.20948616600789
- type: main_score
value: 99.20948616600789
- type: precision
value: 99.1106719367589
- type: recall
value: 99.40711462450594
task:
type: BitextMining
- dataset:
config: rus_Cyrl-twi_Latn
name: MTEB FloresBitextMining (rus_Cyrl-twi_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 82.01581027667984
- type: f1
value: 78.064787822953
- type: main_score
value: 78.064787822953
- type: precision
value: 76.43272186750448
- type: recall
value: 82.01581027667984
task:
type: BitextMining
- dataset:
config: rus_Cyrl-acm_Arab
name: MTEB FloresBitextMining (rus_Cyrl-acm_Arab)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.3201581027668
- type: f1
value: 97.76021080368908
- type: main_score
value: 97.76021080368908
- type: precision
value: 97.48023715415019
- type: recall
value: 98.3201581027668
task:
type: BitextMining
- dataset:
config: rus_Cyrl-bel_Cyrl
name: MTEB FloresBitextMining (rus_Cyrl-bel_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.22134387351778
- type: f1
value: 97.67786561264822
- type: main_score
value: 97.67786561264822
- type: precision
value: 97.4308300395257
- type: recall
value: 98.22134387351778
task:
type: BitextMining
- dataset:
config: rus_Cyrl-eng_Latn
name: MTEB FloresBitextMining (rus_Cyrl-eng_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.70355731225297
- type: f1
value: 99.60474308300395
- type: main_score
value: 99.60474308300395
- type: precision
value: 99.55533596837944
- type: recall
value: 99.70355731225297
task:
type: BitextMining
- dataset:
config: rus_Cyrl-hrv_Latn
name: MTEB FloresBitextMining (rus_Cyrl-hrv_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.1106719367589
- type: f1
value: 98.83069828722002
- type: main_score
value: 98.83069828722002
- type: precision
value: 98.69894598155466
- type: recall
value: 99.1106719367589
task:
type: BitextMining
- dataset:
config: rus_Cyrl-kin_Latn
name: MTEB FloresBitextMining (rus_Cyrl-kin_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 93.37944664031622
- type: f1
value: 91.53162055335969
- type: main_score
value: 91.53162055335969
- type: precision
value: 90.71475625823452
- type: recall
value: 93.37944664031622
task:
type: BitextMining
- dataset:
config: rus_Cyrl-mal_Mlym
name: MTEB FloresBitextMining (rus_Cyrl-mal_Mlym)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.30830039525692
- type: f1
value: 99.07773386034255
- type: main_score
value: 99.07773386034255
- type: precision
value: 98.96245059288538
- type: recall
value: 99.30830039525692
task:
type: BitextMining
- dataset:
config: rus_Cyrl-pes_Arab
name: MTEB FloresBitextMining (rus_Cyrl-pes_Arab)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.71541501976284
- type: f1
value: 98.30368906455863
- type: main_score
value: 98.30368906455863
- type: precision
value: 98.10606060606061
- type: recall
value: 98.71541501976284
task:
type: BitextMining
- dataset:
config: rus_Cyrl-srd_Latn
name: MTEB FloresBitextMining (rus_Cyrl-srd_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 89.03162055335969
- type: f1
value: 86.11048371917937
- type: main_score
value: 86.11048371917937
- type: precision
value: 84.86001317523056
- type: recall
value: 89.03162055335969
task:
type: BitextMining
- dataset:
config: rus_Cyrl-tzm_Tfng
name: MTEB FloresBitextMining (rus_Cyrl-tzm_Tfng)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 12.351778656126482
- type: f1
value: 10.112177999067715
- type: main_score
value: 10.112177999067715
- type: precision
value: 9.53495885438645
- type: recall
value: 12.351778656126482
task:
type: BitextMining
- dataset:
config: rus_Cyrl-acq_Arab
name: MTEB FloresBitextMining (rus_Cyrl-acq_Arab)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.91304347826086
- type: f1
value: 98.55072463768116
- type: main_score
value: 98.55072463768116
- type: precision
value: 98.36956521739131
- type: recall
value: 98.91304347826086
task:
type: BitextMining
- dataset:
config: rus_Cyrl-bem_Latn
name: MTEB FloresBitextMining (rus_Cyrl-bem_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 73.22134387351778
- type: f1
value: 68.30479412989295
- type: main_score
value: 68.30479412989295
- type: precision
value: 66.40073447632736
- type: recall
value: 73.22134387351778
task:
type: BitextMining
- dataset:
config: rus_Cyrl-epo_Latn
name: MTEB FloresBitextMining (rus_Cyrl-epo_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.1106719367589
- type: f1
value: 98.81422924901186
- type: main_score
value: 98.81422924901186
- type: precision
value: 98.66600790513834
- type: recall
value: 99.1106719367589
task:
type: BitextMining
- dataset:
config: rus_Cyrl-hun_Latn
name: MTEB FloresBitextMining (rus_Cyrl-hun_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 96.83794466403161
- type: f1
value: 95.88274044795784
- type: main_score
value: 95.88274044795784
- type: precision
value: 95.45454545454545
- type: recall
value: 96.83794466403161
task:
type: BitextMining
- dataset:
config: rus_Cyrl-kir_Cyrl
name: MTEB FloresBitextMining (rus_Cyrl-kir_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 96.34387351778656
- type: f1
value: 95.49280429715212
- type: main_score
value: 95.49280429715212
- type: precision
value: 95.14163372859026
- type: recall
value: 96.34387351778656
task:
type: BitextMining
- dataset:
config: rus_Cyrl-mar_Deva
name: MTEB FloresBitextMining (rus_Cyrl-mar_Deva)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.71541501976284
- type: f1
value: 98.28722002635047
- type: main_score
value: 98.28722002635047
- type: precision
value: 98.07312252964427
- type: recall
value: 98.71541501976284
task:
type: BitextMining
- dataset:
config: rus_Cyrl-plt_Latn
name: MTEB FloresBitextMining (rus_Cyrl-plt_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 88.04347826086956
- type: f1
value: 85.14328063241106
- type: main_score
value: 85.14328063241106
- type: precision
value: 83.96339168078298
- type: recall
value: 88.04347826086956
task:
type: BitextMining
- dataset:
config: rus_Cyrl-srp_Cyrl
name: MTEB FloresBitextMining (rus_Cyrl-srp_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.40711462450594
- type: f1
value: 99.2094861660079
- type: main_score
value: 99.2094861660079
- type: precision
value: 99.1106719367589
- type: recall
value: 99.40711462450594
task:
type: BitextMining
- dataset:
config: rus_Cyrl-uig_Arab
name: MTEB FloresBitextMining (rus_Cyrl-uig_Arab)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 92.19367588932806
- type: f1
value: 89.98541313758706
- type: main_score
value: 89.98541313758706
- type: precision
value: 89.01021080368906
- type: recall
value: 92.19367588932806
task:
type: BitextMining
- dataset:
config: rus_Cyrl-aeb_Arab
name: MTEB FloresBitextMining (rus_Cyrl-aeb_Arab)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 95.8498023715415
- type: f1
value: 94.63109354413703
- type: main_score
value: 94.63109354413703
- type: precision
value: 94.05467720685111
- type: recall
value: 95.8498023715415
task:
type: BitextMining
- dataset:
config: rus_Cyrl-ben_Beng
name: MTEB FloresBitextMining (rus_Cyrl-ben_Beng)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.40711462450594
- type: f1
value: 99.2094861660079
- type: main_score
value: 99.2094861660079
- type: precision
value: 99.1106719367589
- type: recall
value: 99.40711462450594
task:
type: BitextMining
- dataset:
config: rus_Cyrl-est_Latn
name: MTEB FloresBitextMining (rus_Cyrl-est_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 95.55335968379447
- type: f1
value: 94.2588932806324
- type: main_score
value: 94.2588932806324
- type: precision
value: 93.65118577075098
- type: recall
value: 95.55335968379447
task:
type: BitextMining
- dataset:
config: rus_Cyrl-hye_Armn
name: MTEB FloresBitextMining (rus_Cyrl-hye_Armn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.71541501976284
- type: f1
value: 98.28722002635045
- type: main_score
value: 98.28722002635045
- type: precision
value: 98.07312252964427
- type: recall
value: 98.71541501976284
task:
type: BitextMining
- dataset:
config: rus_Cyrl-kmb_Latn
name: MTEB FloresBitextMining (rus_Cyrl-kmb_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 54.24901185770751
- type: f1
value: 49.46146674116913
- type: main_score
value: 49.46146674116913
- type: precision
value: 47.81033799314432
- type: recall
value: 54.24901185770751
task:
type: BitextMining
- dataset:
config: rus_Cyrl-min_Arab
name: MTEB FloresBitextMining (rus_Cyrl-min_Arab)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 15.810276679841898
- type: f1
value: 13.271207641419332
- type: main_score
value: 13.271207641419332
- type: precision
value: 12.510673148766033
- type: recall
value: 15.810276679841898
task:
type: BitextMining
- dataset:
config: rus_Cyrl-pol_Latn
name: MTEB FloresBitextMining (rus_Cyrl-pol_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.71541501976284
- type: f1
value: 98.32674571805006
- type: main_score
value: 98.32674571805006
- type: precision
value: 98.14723320158103
- type: recall
value: 98.71541501976284
task:
type: BitextMining
- dataset:
config: rus_Cyrl-ssw_Latn
name: MTEB FloresBitextMining (rus_Cyrl-ssw_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 80.8300395256917
- type: f1
value: 76.51717847370023
- type: main_score
value: 76.51717847370023
- type: precision
value: 74.74143610013175
- type: recall
value: 80.8300395256917
task:
type: BitextMining
- dataset:
config: rus_Cyrl-ukr_Cyrl
name: MTEB FloresBitextMining (rus_Cyrl-ukr_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.60474308300395
- type: f1
value: 99.4729907773386
- type: main_score
value: 99.4729907773386
- type: precision
value: 99.40711462450594
- type: recall
value: 99.60474308300395
task:
type: BitextMining
- dataset:
config: rus_Cyrl-afr_Latn
name: MTEB FloresBitextMining (rus_Cyrl-afr_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.1106719367589
- type: f1
value: 98.81422924901186
- type: main_score
value: 98.81422924901186
- type: precision
value: 98.66600790513834
- type: recall
value: 99.1106719367589
task:
type: BitextMining
- dataset:
config: rus_Cyrl-bho_Deva
name: MTEB FloresBitextMining (rus_Cyrl-bho_Deva)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 96.6403162055336
- type: f1
value: 95.56982872200265
- type: main_score
value: 95.56982872200265
- type: precision
value: 95.0592885375494
- type: recall
value: 96.6403162055336
task:
type: BitextMining
- dataset:
config: rus_Cyrl-eus_Latn
name: MTEB FloresBitextMining (rus_Cyrl-eus_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.62845849802372
- type: f1
value: 96.9038208168643
- type: main_score
value: 96.9038208168643
- type: precision
value: 96.55797101449275
- type: recall
value: 97.62845849802372
task:
type: BitextMining
- dataset:
config: rus_Cyrl-ibo_Latn
name: MTEB FloresBitextMining (rus_Cyrl-ibo_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 89.2292490118577
- type: f1
value: 86.35234330886506
- type: main_score
value: 86.35234330886506
- type: precision
value: 85.09881422924902
- type: recall
value: 89.2292490118577
task:
type: BitextMining
- dataset:
config: rus_Cyrl-kmr_Latn
name: MTEB FloresBitextMining (rus_Cyrl-kmr_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 83.49802371541502
- type: f1
value: 79.23630717108978
- type: main_score
value: 79.23630717108978
- type: precision
value: 77.48188405797102
- type: recall
value: 83.49802371541502
task:
type: BitextMining
- dataset:
config: rus_Cyrl-min_Latn
name: MTEB FloresBitextMining (rus_Cyrl-min_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 79.34782608695652
- type: f1
value: 75.31689928429059
- type: main_score
value: 75.31689928429059
- type: precision
value: 73.91519410541149
- type: recall
value: 79.34782608695652
task:
type: BitextMining
- dataset:
config: rus_Cyrl-por_Latn
name: MTEB FloresBitextMining (rus_Cyrl-por_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 96.54150197628458
- type: f1
value: 95.53218520609825
- type: main_score
value: 95.53218520609825
- type: precision
value: 95.07575757575756
- type: recall
value: 96.54150197628458
task:
type: BitextMining
- dataset:
config: rus_Cyrl-sun_Latn
name: MTEB FloresBitextMining (rus_Cyrl-sun_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 93.2806324110672
- type: f1
value: 91.56973461321287
- type: main_score
value: 91.56973461321287
- type: precision
value: 90.84396334890405
- type: recall
value: 93.2806324110672
task:
type: BitextMining
- dataset:
config: rus_Cyrl-umb_Latn
name: MTEB FloresBitextMining (rus_Cyrl-umb_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 51.87747035573123
- type: f1
value: 46.36591778884269
- type: main_score
value: 46.36591778884269
- type: precision
value: 44.57730391234227
- type: recall
value: 51.87747035573123
task:
type: BitextMining
- dataset:
config: rus_Cyrl-ajp_Arab
name: MTEB FloresBitextMining (rus_Cyrl-ajp_Arab)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.71541501976284
- type: f1
value: 98.30368906455863
- type: main_score
value: 98.30368906455863
- type: precision
value: 98.10606060606061
- type: recall
value: 98.71541501976284
task:
type: BitextMining
- dataset:
config: rus_Cyrl-bjn_Arab
name: MTEB FloresBitextMining (rus_Cyrl-bjn_Arab)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 14.82213438735178
- type: f1
value: 12.365434276616856
- type: main_score
value: 12.365434276616856
- type: precision
value: 11.802079517180589
- type: recall
value: 14.82213438735178
task:
type: BitextMining
- dataset:
config: rus_Cyrl-ewe_Latn
name: MTEB FloresBitextMining (rus_Cyrl-ewe_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 71.44268774703558
- type: f1
value: 66.74603174603175
- type: main_score
value: 66.74603174603175
- type: precision
value: 64.99933339607253
- type: recall
value: 71.44268774703558
task:
type: BitextMining
- dataset:
config: rus_Cyrl-ilo_Latn
name: MTEB FloresBitextMining (rus_Cyrl-ilo_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 85.86956521739131
- type: f1
value: 83.00139015960917
- type: main_score
value: 83.00139015960917
- type: precision
value: 81.91411396574439
- type: recall
value: 85.86956521739131
task:
type: BitextMining
- dataset:
config: rus_Cyrl-knc_Arab
name: MTEB FloresBitextMining (rus_Cyrl-knc_Arab)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 14.525691699604742
- type: f1
value: 12.618283715726806
- type: main_score
value: 12.618283715726806
- type: precision
value: 12.048458493742352
- type: recall
value: 14.525691699604742
task:
type: BitextMining
- dataset:
config: rus_Cyrl-mkd_Cyrl
name: MTEB FloresBitextMining (rus_Cyrl-mkd_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.40711462450594
- type: f1
value: 99.22595520421606
- type: main_score
value: 99.22595520421606
- type: precision
value: 99.14361001317523
- type: recall
value: 99.40711462450594
task:
type: BitextMining
- dataset:
config: rus_Cyrl-prs_Arab
name: MTEB FloresBitextMining (rus_Cyrl-prs_Arab)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.30830039525692
- type: f1
value: 99.07773386034255
- type: main_score
value: 99.07773386034255
- type: precision
value: 98.96245059288538
- type: recall
value: 99.30830039525692
task:
type: BitextMining
- dataset:
config: rus_Cyrl-swe_Latn
name: MTEB FloresBitextMining (rus_Cyrl-swe_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.30830039525692
- type: f1
value: 99.07773386034256
- type: main_score
value: 99.07773386034256
- type: precision
value: 98.96245059288538
- type: recall
value: 99.30830039525692
task:
type: BitextMining
- dataset:
config: rus_Cyrl-urd_Arab
name: MTEB FloresBitextMining (rus_Cyrl-urd_Arab)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.61660079051383
- type: f1
value: 98.15546772068511
- type: main_score
value: 98.15546772068511
- type: precision
value: 97.92490118577075
- type: recall
value: 98.61660079051383
task:
type: BitextMining
- dataset:
config: rus_Cyrl-aka_Latn
name: MTEB FloresBitextMining (rus_Cyrl-aka_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 81.02766798418972
- type: f1
value: 76.73277809147375
- type: main_score
value: 76.73277809147375
- type: precision
value: 74.97404165882426
- type: recall
value: 81.02766798418972
task:
type: BitextMining
- dataset:
config: rus_Cyrl-bjn_Latn
name: MTEB FloresBitextMining (rus_Cyrl-bjn_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 86.7588932806324
- type: f1
value: 83.92064566965753
- type: main_score
value: 83.92064566965753
- type: precision
value: 82.83734079929732
- type: recall
value: 86.7588932806324
task:
type: BitextMining
- dataset:
config: rus_Cyrl-fao_Latn
name: MTEB FloresBitextMining (rus_Cyrl-fao_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 88.43873517786561
- type: f1
value: 85.48136645962732
- type: main_score
value: 85.48136645962732
- type: precision
value: 84.23418972332016
- type: recall
value: 88.43873517786561
task:
type: BitextMining
- dataset:
config: rus_Cyrl-ind_Latn
name: MTEB FloresBitextMining (rus_Cyrl-ind_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.01185770750988
- type: f1
value: 98.68247694334651
- type: main_score
value: 98.68247694334651
- type: precision
value: 98.51778656126481
- type: recall
value: 99.01185770750988
task:
type: BitextMining
- dataset:
config: rus_Cyrl-knc_Latn
name: MTEB FloresBitextMining (rus_Cyrl-knc_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 45.8498023715415
- type: f1
value: 40.112030865489366
- type: main_score
value: 40.112030865489366
- type: precision
value: 38.28262440050776
- type: recall
value: 45.8498023715415
task:
type: BitextMining
- dataset:
config: rus_Cyrl-mlt_Latn
name: MTEB FloresBitextMining (rus_Cyrl-mlt_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 93.18181818181817
- type: f1
value: 91.30787690570298
- type: main_score
value: 91.30787690570298
- type: precision
value: 90.4983060417843
- type: recall
value: 93.18181818181817
task:
type: BitextMining
- dataset:
config: rus_Cyrl-quy_Latn
name: MTEB FloresBitextMining (rus_Cyrl-quy_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 62.450592885375485
- type: f1
value: 57.28742975628178
- type: main_score
value: 57.28742975628178
- type: precision
value: 55.56854987623269
- type: recall
value: 62.450592885375485
task:
type: BitextMining
- dataset:
config: rus_Cyrl-swh_Latn
name: MTEB FloresBitextMining (rus_Cyrl-swh_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.3201581027668
- type: f1
value: 97.77667984189723
- type: main_score
value: 97.77667984189723
- type: precision
value: 97.51317523056655
- type: recall
value: 98.3201581027668
task:
type: BitextMining
- dataset:
config: rus_Cyrl-uzn_Latn
name: MTEB FloresBitextMining (rus_Cyrl-uzn_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.12252964426878
- type: f1
value: 97.59081498211933
- type: main_score
value: 97.59081498211933
- type: precision
value: 97.34848484848484
- type: recall
value: 98.12252964426878
task:
type: BitextMining
- dataset:
config: rus_Cyrl-als_Latn
name: MTEB FloresBitextMining (rus_Cyrl-als_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.30830039525692
- type: f1
value: 99.09420289855073
- type: main_score
value: 99.09420289855073
- type: precision
value: 98.99538866930172
- type: recall
value: 99.30830039525692
task:
type: BitextMining
- dataset:
config: rus_Cyrl-bod_Tibt
name: MTEB FloresBitextMining (rus_Cyrl-bod_Tibt)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 11.561264822134387
- type: f1
value: 8.121312045385636
- type: main_score
value: 8.121312045385636
- type: precision
value: 7.350577020893972
- type: recall
value: 11.561264822134387
task:
type: BitextMining
- dataset:
config: rus_Cyrl-fij_Latn
name: MTEB FloresBitextMining (rus_Cyrl-fij_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 72.23320158102767
- type: f1
value: 67.21000233846082
- type: main_score
value: 67.21000233846082
- type: precision
value: 65.3869439739005
- type: recall
value: 72.23320158102767
task:
type: BitextMining
- dataset:
config: rus_Cyrl-isl_Latn
name: MTEB FloresBitextMining (rus_Cyrl-isl_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 91.99604743083005
- type: f1
value: 89.75955204216073
- type: main_score
value: 89.75955204216073
- type: precision
value: 88.7598814229249
- type: recall
value: 91.99604743083005
task:
type: BitextMining
- dataset:
config: rus_Cyrl-kon_Latn
name: MTEB FloresBitextMining (rus_Cyrl-kon_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 81.81818181818183
- type: f1
value: 77.77800098452272
- type: main_score
value: 77.77800098452272
- type: precision
value: 76.1521268586486
- type: recall
value: 81.81818181818183
task:
type: BitextMining
- dataset:
config: rus_Cyrl-mni_Beng
name: MTEB FloresBitextMining (rus_Cyrl-mni_Beng)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 54.74308300395256
- type: f1
value: 48.97285299254615
- type: main_score
value: 48.97285299254615
- type: precision
value: 46.95125742968299
- type: recall
value: 54.74308300395256
task:
type: BitextMining
- dataset:
config: rus_Cyrl-ron_Latn
name: MTEB FloresBitextMining (rus_Cyrl-ron_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.22134387351778
- type: f1
value: 97.64492753623189
- type: main_score
value: 97.64492753623189
- type: precision
value: 97.36495388669302
- type: recall
value: 98.22134387351778
task:
type: BitextMining
- dataset:
config: rus_Cyrl-szl_Latn
name: MTEB FloresBitextMining (rus_Cyrl-szl_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 92.09486166007905
- type: f1
value: 90.10375494071147
- type: main_score
value: 90.10375494071147
- type: precision
value: 89.29606625258798
- type: recall
value: 92.09486166007905
task:
type: BitextMining
- dataset:
config: rus_Cyrl-vec_Latn
name: MTEB FloresBitextMining (rus_Cyrl-vec_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 92.4901185770751
- type: f1
value: 90.51430453604365
- type: main_score
value: 90.51430453604365
- type: precision
value: 89.69367588932808
- type: recall
value: 92.4901185770751
task:
type: BitextMining
- dataset:
config: rus_Cyrl-amh_Ethi
name: MTEB FloresBitextMining (rus_Cyrl-amh_Ethi)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.82608695652173
- type: f1
value: 97.11791831357048
- type: main_score
value: 97.11791831357048
- type: precision
value: 96.77206851119894
- type: recall
value: 97.82608695652173
task:
type: BitextMining
- dataset:
config: rus_Cyrl-bos_Latn
name: MTEB FloresBitextMining (rus_Cyrl-bos_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.91304347826086
- type: f1
value: 98.55072463768116
- type: main_score
value: 98.55072463768116
- type: precision
value: 98.36956521739131
- type: recall
value: 98.91304347826086
task:
type: BitextMining
- dataset:
config: rus_Cyrl-fin_Latn
name: MTEB FloresBitextMining (rus_Cyrl-fin_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 95.65217391304348
- type: f1
value: 94.4235836627141
- type: main_score
value: 94.4235836627141
- type: precision
value: 93.84881422924902
- type: recall
value: 95.65217391304348
task:
type: BitextMining
- dataset:
config: rus_Cyrl-ita_Latn
name: MTEB FloresBitextMining (rus_Cyrl-ita_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.91304347826086
- type: f1
value: 98.55072463768117
- type: main_score
value: 98.55072463768117
- type: precision
value: 98.36956521739131
- type: recall
value: 98.91304347826086
task:
type: BitextMining
- dataset:
config: rus_Cyrl-kor_Hang
name: MTEB FloresBitextMining (rus_Cyrl-kor_Hang)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 95.55335968379447
- type: f1
value: 94.15349143610013
- type: main_score
value: 94.15349143610013
- type: precision
value: 93.49472990777339
- type: recall
value: 95.55335968379447
task:
type: BitextMining
- dataset:
config: rus_Cyrl-mos_Latn
name: MTEB FloresBitextMining (rus_Cyrl-mos_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 43.67588932806324
- type: f1
value: 38.84849721190082
- type: main_score
value: 38.84849721190082
- type: precision
value: 37.43294462099682
- type: recall
value: 43.67588932806324
task:
type: BitextMining
- dataset:
config: rus_Cyrl-run_Latn
name: MTEB FloresBitextMining (rus_Cyrl-run_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 90.21739130434783
- type: f1
value: 87.37483530961792
- type: main_score
value: 87.37483530961792
- type: precision
value: 86.07872200263506
- type: recall
value: 90.21739130434783
task:
type: BitextMining
- dataset:
config: rus_Cyrl-tam_Taml
name: MTEB FloresBitextMining (rus_Cyrl-tam_Taml)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.40711462450594
- type: f1
value: 99.2094861660079
- type: main_score
value: 99.2094861660079
- type: precision
value: 99.1106719367589
- type: recall
value: 99.40711462450594
task:
type: BitextMining
- dataset:
config: rus_Cyrl-vie_Latn
name: MTEB FloresBitextMining (rus_Cyrl-vie_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.03557312252964
- type: f1
value: 96.13636363636364
- type: main_score
value: 96.13636363636364
- type: precision
value: 95.70981554677206
- type: recall
value: 97.03557312252964
task:
type: BitextMining
- dataset:
config: rus_Cyrl-apc_Arab
name: MTEB FloresBitextMining (rus_Cyrl-apc_Arab)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.12252964426878
- type: f1
value: 97.49670619235836
- type: main_score
value: 97.49670619235836
- type: precision
value: 97.18379446640316
- type: recall
value: 98.12252964426878
task:
type: BitextMining
- dataset:
config: rus_Cyrl-bug_Latn
name: MTEB FloresBitextMining (rus_Cyrl-bug_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 67.29249011857708
- type: f1
value: 62.09268717667927
- type: main_score
value: 62.09268717667927
- type: precision
value: 60.28554009748714
- type: recall
value: 67.29249011857708
task:
type: BitextMining
- dataset:
config: rus_Cyrl-fon_Latn
name: MTEB FloresBitextMining (rus_Cyrl-fon_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 63.43873517786561
- type: f1
value: 57.66660107569199
- type: main_score
value: 57.66660107569199
- type: precision
value: 55.66676396919363
- type: recall
value: 63.43873517786561
task:
type: BitextMining
- dataset:
config: rus_Cyrl-jav_Latn
name: MTEB FloresBitextMining (rus_Cyrl-jav_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 94.46640316205533
- type: f1
value: 92.89384528514964
- type: main_score
value: 92.89384528514964
- type: precision
value: 92.19367588932806
- type: recall
value: 94.46640316205533
task:
type: BitextMining
- dataset:
config: rus_Cyrl-lao_Laoo
name: MTEB FloresBitextMining (rus_Cyrl-lao_Laoo)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.23320158102767
- type: f1
value: 96.40974967061922
- type: main_score
value: 96.40974967061922
- type: precision
value: 96.034255599473
- type: recall
value: 97.23320158102767
task:
type: BitextMining
- dataset:
config: rus_Cyrl-mri_Latn
name: MTEB FloresBitextMining (rus_Cyrl-mri_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 76.77865612648222
- type: f1
value: 73.11286539547409
- type: main_score
value: 73.11286539547409
- type: precision
value: 71.78177214337046
- type: recall
value: 76.77865612648222
task:
type: BitextMining
- dataset:
config: rus_Cyrl-taq_Latn
name: MTEB FloresBitextMining (rus_Cyrl-taq_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 41.99604743083004
- type: f1
value: 37.25127063318763
- type: main_score
value: 37.25127063318763
- type: precision
value: 35.718929186985726
- type: recall
value: 41.99604743083004
task:
type: BitextMining
- dataset:
config: rus_Cyrl-war_Latn
name: MTEB FloresBitextMining (rus_Cyrl-war_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 95.55335968379447
- type: f1
value: 94.1699604743083
- type: main_score
value: 94.1699604743083
- type: precision
value: 93.52766798418972
- type: recall
value: 95.55335968379447
task:
type: BitextMining
- dataset:
config: rus_Cyrl-arb_Arab
name: MTEB FloresBitextMining (rus_Cyrl-arb_Arab)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.60474308300395
- type: f1
value: 99.4729907773386
- type: main_score
value: 99.4729907773386
- type: precision
value: 99.40711462450594
- type: recall
value: 99.60474308300395
task:
type: BitextMining
- dataset:
config: rus_Cyrl-bul_Cyrl
name: MTEB FloresBitextMining (rus_Cyrl-bul_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.70355731225297
- type: f1
value: 99.60474308300395
- type: main_score
value: 99.60474308300395
- type: precision
value: 99.55533596837944
- type: recall
value: 99.70355731225297
task:
type: BitextMining
- dataset:
config: rus_Cyrl-fra_Latn
name: MTEB FloresBitextMining (rus_Cyrl-fra_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.60474308300395
- type: f1
value: 99.47299077733861
- type: main_score
value: 99.47299077733861
- type: precision
value: 99.40711462450594
- type: recall
value: 99.60474308300395
task:
type: BitextMining
- dataset:
config: rus_Cyrl-jpn_Jpan
name: MTEB FloresBitextMining (rus_Cyrl-jpn_Jpan)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 96.44268774703558
- type: f1
value: 95.30632411067194
- type: main_score
value: 95.30632411067194
- type: precision
value: 94.76284584980237
- type: recall
value: 96.44268774703558
task:
type: BitextMining
- dataset:
config: rus_Cyrl-lij_Latn
name: MTEB FloresBitextMining (rus_Cyrl-lij_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 90.21739130434783
- type: f1
value: 87.4703557312253
- type: main_score
value: 87.4703557312253
- type: precision
value: 86.29611330698287
- type: recall
value: 90.21739130434783
task:
type: BitextMining
- dataset:
config: rus_Cyrl-mya_Mymr
name: MTEB FloresBitextMining (rus_Cyrl-mya_Mymr)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.02371541501977
- type: f1
value: 97.364953886693
- type: main_score
value: 97.364953886693
- type: precision
value: 97.03557312252964
- type: recall
value: 98.02371541501977
task:
type: BitextMining
- dataset:
config: rus_Cyrl-sag_Latn
name: MTEB FloresBitextMining (rus_Cyrl-sag_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 54.841897233201585
- type: f1
value: 49.61882037503349
- type: main_score
value: 49.61882037503349
- type: precision
value: 47.831968755881796
- type: recall
value: 54.841897233201585
task:
type: BitextMining
- dataset:
config: rus_Cyrl-taq_Tfng
name: MTEB FloresBitextMining (rus_Cyrl-taq_Tfng)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 15.316205533596838
- type: f1
value: 11.614836360389717
- type: main_score
value: 11.614836360389717
- type: precision
value: 10.741446193235223
- type: recall
value: 15.316205533596838
task:
type: BitextMining
- dataset:
config: rus_Cyrl-wol_Latn
name: MTEB FloresBitextMining (rus_Cyrl-wol_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 67.88537549407114
- type: f1
value: 62.2536417249856
- type: main_score
value: 62.2536417249856
- type: precision
value: 60.27629128666678
- type: recall
value: 67.88537549407114
task:
type: BitextMining
- dataset:
config: rus_Cyrl-arb_Latn
name: MTEB FloresBitextMining (rus_Cyrl-arb_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 27.766798418972332
- type: f1
value: 23.39674889624077
- type: main_score
value: 23.39674889624077
- type: precision
value: 22.28521155585345
- type: recall
value: 27.766798418972332
task:
type: BitextMining
- dataset:
config: rus_Cyrl-cat_Latn
name: MTEB FloresBitextMining (rus_Cyrl-cat_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.23320158102767
- type: f1
value: 96.42151326933936
- type: main_score
value: 96.42151326933936
- type: precision
value: 96.04743083003953
- type: recall
value: 97.23320158102767
task:
type: BitextMining
- dataset:
config: rus_Cyrl-fur_Latn
name: MTEB FloresBitextMining (rus_Cyrl-fur_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 88.63636363636364
- type: f1
value: 85.80792396009788
- type: main_score
value: 85.80792396009788
- type: precision
value: 84.61508901726293
- type: recall
value: 88.63636363636364
task:
type: BitextMining
- dataset:
config: rus_Cyrl-kab_Latn
name: MTEB FloresBitextMining (rus_Cyrl-kab_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 48.12252964426877
- type: f1
value: 43.05387582971066
- type: main_score
value: 43.05387582971066
- type: precision
value: 41.44165117538212
- type: recall
value: 48.12252964426877
task:
type: BitextMining
- dataset:
config: rus_Cyrl-lim_Latn
name: MTEB FloresBitextMining (rus_Cyrl-lim_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 81.81818181818183
- type: f1
value: 77.81676163099087
- type: main_score
value: 77.81676163099087
- type: precision
value: 76.19565217391305
- type: recall
value: 81.81818181818183
task:
type: BitextMining
- dataset:
config: rus_Cyrl-nld_Latn
name: MTEB FloresBitextMining (rus_Cyrl-nld_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.33201581027669
- type: f1
value: 96.4756258234519
- type: main_score
value: 96.4756258234519
- type: precision
value: 96.06389986824769
- type: recall
value: 97.33201581027669
task:
type: BitextMining
- dataset:
config: rus_Cyrl-san_Deva
name: MTEB FloresBitextMining (rus_Cyrl-san_Deva)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 93.47826086956522
- type: f1
value: 91.70289855072463
- type: main_score
value: 91.70289855072463
- type: precision
value: 90.9370882740448
- type: recall
value: 93.47826086956522
task:
type: BitextMining
- dataset:
config: rus_Cyrl-tat_Cyrl
name: MTEB FloresBitextMining (rus_Cyrl-tat_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.72727272727273
- type: f1
value: 97.00263504611331
- type: main_score
value: 97.00263504611331
- type: precision
value: 96.65678524374177
- type: recall
value: 97.72727272727273
task:
type: BitextMining
- dataset:
config: rus_Cyrl-xho_Latn
name: MTEB FloresBitextMining (rus_Cyrl-xho_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 93.08300395256917
- type: f1
value: 91.12977602108036
- type: main_score
value: 91.12977602108036
- type: precision
value: 90.22562582345192
- type: recall
value: 93.08300395256917
task:
type: BitextMining
- dataset:
config: rus_Cyrl-ars_Arab
name: MTEB FloresBitextMining (rus_Cyrl-ars_Arab)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.40711462450594
- type: f1
value: 99.2094861660079
- type: main_score
value: 99.2094861660079
- type: precision
value: 99.1106719367589
- type: recall
value: 99.40711462450594
task:
type: BitextMining
- dataset:
config: rus_Cyrl-ceb_Latn
name: MTEB FloresBitextMining (rus_Cyrl-ceb_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 95.65217391304348
- type: f1
value: 94.3544137022398
- type: main_score
value: 94.3544137022398
- type: precision
value: 93.76646903820817
- type: recall
value: 95.65217391304348
task:
type: BitextMining
- dataset:
config: rus_Cyrl-fuv_Latn
name: MTEB FloresBitextMining (rus_Cyrl-fuv_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 51.18577075098815
- type: f1
value: 44.5990252610806
- type: main_score
value: 44.5990252610806
- type: precision
value: 42.34331599450177
- type: recall
value: 51.18577075098815
task:
type: BitextMining
- dataset:
config: rus_Cyrl-kac_Latn
name: MTEB FloresBitextMining (rus_Cyrl-kac_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 46.93675889328063
- type: f1
value: 41.79004018701787
- type: main_score
value: 41.79004018701787
- type: precision
value: 40.243355662392624
- type: recall
value: 46.93675889328063
task:
type: BitextMining
- dataset:
config: rus_Cyrl-lin_Latn
name: MTEB FloresBitextMining (rus_Cyrl-lin_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 91.50197628458498
- type: f1
value: 89.1205533596838
- type: main_score
value: 89.1205533596838
- type: precision
value: 88.07147562582345
- type: recall
value: 91.50197628458498
task:
type: BitextMining
- dataset:
config: rus_Cyrl-nno_Latn
name: MTEB FloresBitextMining (rus_Cyrl-nno_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.81422924901186
- type: f1
value: 98.41897233201581
- type: main_score
value: 98.41897233201581
- type: precision
value: 98.22134387351778
- type: recall
value: 98.81422924901186
task:
type: BitextMining
- dataset:
config: rus_Cyrl-sat_Olck
name: MTEB FloresBitextMining (rus_Cyrl-sat_Olck)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 2.371541501976284
- type: f1
value: 1.0726274943087382
- type: main_score
value: 1.0726274943087382
- type: precision
value: 0.875279634748803
- type: recall
value: 2.371541501976284
task:
type: BitextMining
- dataset:
config: rus_Cyrl-tel_Telu
name: MTEB FloresBitextMining (rus_Cyrl-tel_Telu)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.01185770750988
- type: f1
value: 98.68247694334651
- type: main_score
value: 98.68247694334651
- type: precision
value: 98.51778656126481
- type: recall
value: 99.01185770750988
task:
type: BitextMining
- dataset:
config: rus_Cyrl-ydd_Hebr
name: MTEB FloresBitextMining (rus_Cyrl-ydd_Hebr)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 89.42687747035573
- type: f1
value: 86.47609636740073
- type: main_score
value: 86.47609636740073
- type: precision
value: 85.13669301712781
- type: recall
value: 89.42687747035573
task:
type: BitextMining
- dataset:
config: rus_Cyrl-ary_Arab
name: MTEB FloresBitextMining (rus_Cyrl-ary_Arab)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 89.82213438735178
- type: f1
value: 87.04545454545456
- type: main_score
value: 87.04545454545456
- type: precision
value: 85.76910408432148
- type: recall
value: 89.82213438735178
task:
type: BitextMining
- dataset:
config: rus_Cyrl-ces_Latn
name: MTEB FloresBitextMining (rus_Cyrl-ces_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.2094861660079
- type: f1
value: 98.9459815546772
- type: main_score
value: 98.9459815546772
- type: precision
value: 98.81422924901186
- type: recall
value: 99.2094861660079
task:
type: BitextMining
- dataset:
config: rus_Cyrl-gaz_Latn
name: MTEB FloresBitextMining (rus_Cyrl-gaz_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 64.9209486166008
- type: f1
value: 58.697458119394874
- type: main_score
value: 58.697458119394874
- type: precision
value: 56.43402189597842
- type: recall
value: 64.9209486166008
task:
type: BitextMining
- dataset:
config: rus_Cyrl-kam_Latn
name: MTEB FloresBitextMining (rus_Cyrl-kam_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 59.18972332015811
- type: f1
value: 53.19031511966295
- type: main_score
value: 53.19031511966295
- type: precision
value: 51.08128357343655
- type: recall
value: 59.18972332015811
task:
type: BitextMining
- dataset:
config: rus_Cyrl-lit_Latn
name: MTEB FloresBitextMining (rus_Cyrl-lit_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 96.54150197628458
- type: f1
value: 95.5368906455863
- type: main_score
value: 95.5368906455863
- type: precision
value: 95.0592885375494
- type: recall
value: 96.54150197628458
task:
type: BitextMining
- dataset:
config: rus_Cyrl-nob_Latn
name: MTEB FloresBitextMining (rus_Cyrl-nob_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.12252964426878
- type: f1
value: 97.51317523056655
- type: main_score
value: 97.51317523056655
- type: precision
value: 97.2167325428195
- type: recall
value: 98.12252964426878
task:
type: BitextMining
- dataset:
config: rus_Cyrl-scn_Latn
name: MTEB FloresBitextMining (rus_Cyrl-scn_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 84.0909090909091
- type: f1
value: 80.37000439174352
- type: main_score
value: 80.37000439174352
- type: precision
value: 78.83994628559846
- type: recall
value: 84.0909090909091
task:
type: BitextMining
- dataset:
config: rus_Cyrl-tgk_Cyrl
name: MTEB FloresBitextMining (rus_Cyrl-tgk_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 92.68774703557312
- type: f1
value: 90.86344814605684
- type: main_score
value: 90.86344814605684
- type: precision
value: 90.12516469038208
- type: recall
value: 92.68774703557312
task:
type: BitextMining
- dataset:
config: rus_Cyrl-yor_Latn
name: MTEB FloresBitextMining (rus_Cyrl-yor_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 72.13438735177866
- type: f1
value: 66.78759646150951
- type: main_score
value: 66.78759646150951
- type: precision
value: 64.85080192096002
- type: recall
value: 72.13438735177866
task:
type: BitextMining
- dataset:
config: rus_Cyrl-arz_Arab
name: MTEB FloresBitextMining (rus_Cyrl-arz_Arab)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.02371541501977
- type: f1
value: 97.364953886693
- type: main_score
value: 97.364953886693
- type: precision
value: 97.03557312252964
- type: recall
value: 98.02371541501977
task:
type: BitextMining
- dataset:
config: rus_Cyrl-cjk_Latn
name: MTEB FloresBitextMining (rus_Cyrl-cjk_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 51.976284584980235
- type: f1
value: 46.468762353149714
- type: main_score
value: 46.468762353149714
- type: precision
value: 44.64073366247278
- type: recall
value: 51.976284584980235
task:
type: BitextMining
- dataset:
config: rus_Cyrl-gla_Latn
name: MTEB FloresBitextMining (rus_Cyrl-gla_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 79.74308300395256
- type: f1
value: 75.55611165294958
- type: main_score
value: 75.55611165294958
- type: precision
value: 73.95033408620365
- type: recall
value: 79.74308300395256
task:
type: BitextMining
- dataset:
config: rus_Cyrl-kan_Knda
name: MTEB FloresBitextMining (rus_Cyrl-kan_Knda)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.2094861660079
- type: f1
value: 98.96245059288538
- type: main_score
value: 98.96245059288538
- type: precision
value: 98.84716732542819
- type: recall
value: 99.2094861660079
task:
type: BitextMining
- dataset:
config: rus_Cyrl-lmo_Latn
name: MTEB FloresBitextMining (rus_Cyrl-lmo_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 82.41106719367589
- type: f1
value: 78.56413514022209
- type: main_score
value: 78.56413514022209
- type: precision
value: 77.15313068573938
- type: recall
value: 82.41106719367589
task:
type: BitextMining
- dataset:
config: rus_Cyrl-npi_Deva
name: MTEB FloresBitextMining (rus_Cyrl-npi_Deva)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.71541501976284
- type: f1
value: 98.3201581027668
- type: main_score
value: 98.3201581027668
- type: precision
value: 98.12252964426878
- type: recall
value: 98.71541501976284
task:
type: BitextMining
- dataset:
config: rus_Cyrl-shn_Mymr
name: MTEB FloresBitextMining (rus_Cyrl-shn_Mymr)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 57.11462450592886
- type: f1
value: 51.51361369197337
- type: main_score
value: 51.51361369197337
- type: precision
value: 49.71860043649573
- type: recall
value: 57.11462450592886
task:
type: BitextMining
- dataset:
config: rus_Cyrl-tgl_Latn
name: MTEB FloresBitextMining (rus_Cyrl-tgl_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.82608695652173
- type: f1
value: 97.18379446640316
- type: main_score
value: 97.18379446640316
- type: precision
value: 96.88735177865613
- type: recall
value: 97.82608695652173
task:
type: BitextMining
- dataset:
config: rus_Cyrl-yue_Hant
name: MTEB FloresBitextMining (rus_Cyrl-yue_Hant)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.30830039525692
- type: f1
value: 99.09420289855072
- type: main_score
value: 99.09420289855072
- type: precision
value: 98.9953886693017
- type: recall
value: 99.30830039525692
task:
type: BitextMining
- dataset:
config: rus_Cyrl-asm_Beng
name: MTEB FloresBitextMining (rus_Cyrl-asm_Beng)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 95.55335968379447
- type: f1
value: 94.16007905138339
- type: main_score
value: 94.16007905138339
- type: precision
value: 93.50296442687747
- type: recall
value: 95.55335968379447
task:
type: BitextMining
- dataset:
config: rus_Cyrl-ckb_Arab
name: MTEB FloresBitextMining (rus_Cyrl-ckb_Arab)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 92.88537549407114
- type: f1
value: 90.76745718050066
- type: main_score
value: 90.76745718050066
- type: precision
value: 89.80072463768116
- type: recall
value: 92.88537549407114
task:
type: BitextMining
- dataset:
config: rus_Cyrl-gle_Latn
name: MTEB FloresBitextMining (rus_Cyrl-gle_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 91.699604743083
- type: f1
value: 89.40899680030115
- type: main_score
value: 89.40899680030115
- type: precision
value: 88.40085638998683
- type: recall
value: 91.699604743083
task:
type: BitextMining
- dataset:
config: rus_Cyrl-kas_Arab
name: MTEB FloresBitextMining (rus_Cyrl-kas_Arab)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 88.3399209486166
- type: f1
value: 85.14351590438548
- type: main_score
value: 85.14351590438548
- type: precision
value: 83.72364953886692
- type: recall
value: 88.3399209486166
task:
type: BitextMining
- dataset:
config: rus_Cyrl-ltg_Latn
name: MTEB FloresBitextMining (rus_Cyrl-ltg_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 83.399209486166
- type: f1
value: 79.88408934061107
- type: main_score
value: 79.88408934061107
- type: precision
value: 78.53794509179885
- type: recall
value: 83.399209486166
task:
type: BitextMining
- dataset:
config: rus_Cyrl-nso_Latn
name: MTEB FloresBitextMining (rus_Cyrl-nso_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 91.20553359683794
- type: f1
value: 88.95406635525212
- type: main_score
value: 88.95406635525212
- type: precision
value: 88.01548089591567
- type: recall
value: 91.20553359683794
task:
type: BitextMining
- dataset:
config: rus_Cyrl-sin_Sinh
name: MTEB FloresBitextMining (rus_Cyrl-sin_Sinh)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.91304347826086
- type: f1
value: 98.56719367588933
- type: main_score
value: 98.56719367588933
- type: precision
value: 98.40250329380763
- type: recall
value: 98.91304347826086
task:
type: BitextMining
- dataset:
config: rus_Cyrl-tha_Thai
name: MTEB FloresBitextMining (rus_Cyrl-tha_Thai)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 95.94861660079052
- type: f1
value: 94.66403162055336
- type: main_score
value: 94.66403162055336
- type: precision
value: 94.03820816864295
- type: recall
value: 95.94861660079052
task:
type: BitextMining
- dataset:
config: rus_Cyrl-zho_Hans
name: MTEB FloresBitextMining (rus_Cyrl-zho_Hans)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.4308300395257
- type: f1
value: 96.5909090909091
- type: main_score
value: 96.5909090909091
- type: precision
value: 96.17918313570487
- type: recall
value: 97.4308300395257
task:
type: BitextMining
- dataset:
config: rus_Cyrl-ast_Latn
name: MTEB FloresBitextMining (rus_Cyrl-ast_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 94.46640316205533
- type: f1
value: 92.86890645586297
- type: main_score
value: 92.86890645586297
- type: precision
value: 92.14756258234519
- type: recall
value: 94.46640316205533
task:
type: BitextMining
- dataset:
config: rus_Cyrl-crh_Latn
name: MTEB FloresBitextMining (rus_Cyrl-crh_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 94.66403162055336
- type: f1
value: 93.2663592446201
- type: main_score
value: 93.2663592446201
- type: precision
value: 92.66716073781292
- type: recall
value: 94.66403162055336
task:
type: BitextMining
- dataset:
config: rus_Cyrl-glg_Latn
name: MTEB FloresBitextMining (rus_Cyrl-glg_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.81422924901186
- type: f1
value: 98.46837944664031
- type: main_score
value: 98.46837944664031
- type: precision
value: 98.3201581027668
- type: recall
value: 98.81422924901186
task:
type: BitextMining
- dataset:
config: rus_Cyrl-kas_Deva
name: MTEB FloresBitextMining (rus_Cyrl-kas_Deva)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 69.1699604743083
- type: f1
value: 63.05505292906477
- type: main_score
value: 63.05505292906477
- type: precision
value: 60.62594108789761
- type: recall
value: 69.1699604743083
task:
type: BitextMining
- dataset:
config: rus_Cyrl-ltz_Latn
name: MTEB FloresBitextMining (rus_Cyrl-ltz_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 91.40316205533597
- type: f1
value: 89.26571616789009
- type: main_score
value: 89.26571616789009
- type: precision
value: 88.40179747788443
- type: recall
value: 91.40316205533597
task:
type: BitextMining
- dataset:
config: rus_Cyrl-nus_Latn
name: MTEB FloresBitextMining (rus_Cyrl-nus_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 38.93280632411067
- type: f1
value: 33.98513032905371
- type: main_score
value: 33.98513032905371
- type: precision
value: 32.56257884802308
- type: recall
value: 38.93280632411067
task:
type: BitextMining
- dataset:
config: rus_Cyrl-slk_Latn
name: MTEB FloresBitextMining (rus_Cyrl-slk_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.02371541501977
- type: f1
value: 97.42094861660078
- type: main_score
value: 97.42094861660078
- type: precision
value: 97.14262187088273
- type: recall
value: 98.02371541501977
task:
type: BitextMining
- dataset:
config: rus_Cyrl-tir_Ethi
name: MTEB FloresBitextMining (rus_Cyrl-tir_Ethi)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 91.30434782608695
- type: f1
value: 88.78129117259552
- type: main_score
value: 88.78129117259552
- type: precision
value: 87.61528326745717
- type: recall
value: 91.30434782608695
task:
type: BitextMining
- dataset:
config: rus_Cyrl-zho_Hant
name: MTEB FloresBitextMining (rus_Cyrl-zho_Hant)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.1106719367589
- type: f1
value: 98.81422924901186
- type: main_score
value: 98.81422924901186
- type: precision
value: 98.66600790513834
- type: recall
value: 99.1106719367589
task:
type: BitextMining
- dataset:
config: rus_Cyrl-awa_Deva
name: MTEB FloresBitextMining (rus_Cyrl-awa_Deva)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.12252964426878
- type: f1
value: 97.70092226613966
- type: main_score
value: 97.70092226613966
- type: precision
value: 97.50494071146245
- type: recall
value: 98.12252964426878
task:
type: BitextMining
- dataset:
config: rus_Cyrl-cym_Latn
name: MTEB FloresBitextMining (rus_Cyrl-cym_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 95.94861660079052
- type: f1
value: 94.74308300395256
- type: main_score
value: 94.74308300395256
- type: precision
value: 94.20289855072464
- type: recall
value: 95.94861660079052
task:
type: BitextMining
- dataset:
config: rus_Cyrl-grn_Latn
name: MTEB FloresBitextMining (rus_Cyrl-grn_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 77.96442687747036
- type: f1
value: 73.64286789187975
- type: main_score
value: 73.64286789187975
- type: precision
value: 71.99324893260821
- type: recall
value: 77.96442687747036
task:
type: BitextMining
- dataset:
config: rus_Cyrl-kat_Geor
name: MTEB FloresBitextMining (rus_Cyrl-kat_Geor)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.91304347826086
- type: f1
value: 98.56719367588933
- type: main_score
value: 98.56719367588933
- type: precision
value: 98.40250329380764
- type: recall
value: 98.91304347826086
task:
type: BitextMining
- dataset:
config: rus_Cyrl-lua_Latn
name: MTEB FloresBitextMining (rus_Cyrl-lua_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 72.03557312252964
- type: f1
value: 67.23928163404449
- type: main_score
value: 67.23928163404449
- type: precision
value: 65.30797101449275
- type: recall
value: 72.03557312252964
task:
type: BitextMining
- dataset:
config: rus_Cyrl-nya_Latn
name: MTEB FloresBitextMining (rus_Cyrl-nya_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 92.29249011857708
- type: f1
value: 90.0494071146245
- type: main_score
value: 90.0494071146245
- type: precision
value: 89.04808959156786
- type: recall
value: 92.29249011857708
task:
type: BitextMining
- dataset:
config: rus_Cyrl-slv_Latn
name: MTEB FloresBitextMining (rus_Cyrl-slv_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.71541501976284
- type: f1
value: 98.30368906455863
- type: main_score
value: 98.30368906455863
- type: precision
value: 98.10606060606061
- type: recall
value: 98.71541501976284
task:
type: BitextMining
- dataset:
config: rus_Cyrl-tpi_Latn
name: MTEB FloresBitextMining (rus_Cyrl-tpi_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 80.53359683794467
- type: f1
value: 76.59481822525301
- type: main_score
value: 76.59481822525301
- type: precision
value: 75.12913223140497
- type: recall
value: 80.53359683794467
task:
type: BitextMining
- dataset:
config: rus_Cyrl-zsm_Latn
name: MTEB FloresBitextMining (rus_Cyrl-zsm_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.33201581027669
- type: f1
value: 96.58620365142104
- type: main_score
value: 96.58620365142104
- type: precision
value: 96.26152832674572
- type: recall
value: 97.33201581027669
task:
type: BitextMining
- dataset:
config: rus_Cyrl-ayr_Latn
name: MTEB FloresBitextMining (rus_Cyrl-ayr_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 45.55335968379446
- type: f1
value: 40.13076578531388
- type: main_score
value: 40.13076578531388
- type: precision
value: 38.398064362362355
- type: recall
value: 45.55335968379446
task:
type: BitextMining
- dataset:
config: rus_Cyrl-dan_Latn
name: MTEB FloresBitextMining (rus_Cyrl-dan_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.01185770750988
- type: f1
value: 98.68247694334651
- type: main_score
value: 98.68247694334651
- type: precision
value: 98.51778656126481
- type: recall
value: 99.01185770750988
task:
type: BitextMining
- dataset:
config: rus_Cyrl-guj_Gujr
name: MTEB FloresBitextMining (rus_Cyrl-guj_Gujr)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.01185770750988
- type: f1
value: 98.68247694334651
- type: main_score
value: 98.68247694334651
- type: precision
value: 98.51778656126481
- type: recall
value: 99.01185770750988
task:
type: BitextMining
- dataset:
config: rus_Cyrl-kaz_Cyrl
name: MTEB FloresBitextMining (rus_Cyrl-kaz_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.81422924901186
- type: f1
value: 98.43544137022398
- type: main_score
value: 98.43544137022398
- type: precision
value: 98.25428194993412
- type: recall
value: 98.81422924901186
task:
type: BitextMining
- dataset:
config: rus_Cyrl-lug_Latn
name: MTEB FloresBitextMining (rus_Cyrl-lug_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 82.21343873517787
- type: f1
value: 77.97485726833554
- type: main_score
value: 77.97485726833554
- type: precision
value: 76.22376717485415
- type: recall
value: 82.21343873517787
task:
type: BitextMining
- dataset:
config: rus_Cyrl-oci_Latn
name: MTEB FloresBitextMining (rus_Cyrl-oci_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 93.87351778656127
- type: f1
value: 92.25319969885187
- type: main_score
value: 92.25319969885187
- type: precision
value: 91.5638528138528
- type: recall
value: 93.87351778656127
task:
type: BitextMining
- dataset:
config: rus_Cyrl-smo_Latn
name: MTEB FloresBitextMining (rus_Cyrl-smo_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 84.88142292490119
- type: f1
value: 81.24364765669114
- type: main_score
value: 81.24364765669114
- type: precision
value: 79.69991416137661
- type: recall
value: 84.88142292490119
task:
type: BitextMining
- dataset:
config: rus_Cyrl-tsn_Latn
name: MTEB FloresBitextMining (rus_Cyrl-tsn_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 87.05533596837944
- type: f1
value: 83.90645586297761
- type: main_score
value: 83.90645586297761
- type: precision
value: 82.56752305665349
- type: recall
value: 87.05533596837944
task:
type: BitextMining
- dataset:
config: rus_Cyrl-zul_Latn
name: MTEB FloresBitextMining (rus_Cyrl-zul_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 95.15810276679841
- type: f1
value: 93.77140974967062
- type: main_score
value: 93.77140974967062
- type: precision
value: 93.16534914361002
- type: recall
value: 95.15810276679841
task:
type: BitextMining
- dataset:
config: rus_Cyrl-azb_Arab
name: MTEB FloresBitextMining (rus_Cyrl-azb_Arab)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 81.91699604743083
- type: f1
value: 77.18050065876152
- type: main_score
value: 77.18050065876152
- type: precision
value: 75.21519543258673
- type: recall
value: 81.91699604743083
task:
type: BitextMining
- dataset:
config: rus_Cyrl-deu_Latn
name: MTEB FloresBitextMining (rus_Cyrl-deu_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.50592885375494
- type: f1
value: 99.34123847167325
- type: main_score
value: 99.34123847167325
- type: precision
value: 99.2588932806324
- type: recall
value: 99.50592885375494
task:
type: BitextMining
- dataset:
config: rus_Cyrl-hat_Latn
name: MTEB FloresBitextMining (rus_Cyrl-hat_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 91.00790513833992
- type: f1
value: 88.69126043039086
- type: main_score
value: 88.69126043039086
- type: precision
value: 87.75774044795784
- type: recall
value: 91.00790513833992
task:
type: BitextMining
- dataset:
config: rus_Cyrl-kbp_Latn
name: MTEB FloresBitextMining (rus_Cyrl-kbp_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 47.233201581027664
- type: f1
value: 43.01118618096943
- type: main_score
value: 43.01118618096943
- type: precision
value: 41.739069205043556
- type: recall
value: 47.233201581027664
task:
type: BitextMining
- dataset:
config: rus_Cyrl-luo_Latn
name: MTEB FloresBitextMining (rus_Cyrl-luo_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 60.47430830039525
- type: f1
value: 54.83210565429816
- type: main_score
value: 54.83210565429816
- type: precision
value: 52.81630744284779
- type: recall
value: 60.47430830039525
task:
type: BitextMining
- dataset:
config: rus_Cyrl-ory_Orya
name: MTEB FloresBitextMining (rus_Cyrl-ory_Orya)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.1106719367589
- type: f1
value: 98.83069828722003
- type: main_score
value: 98.83069828722003
- type: precision
value: 98.69894598155467
- type: recall
value: 99.1106719367589
task:
type: BitextMining
- dataset:
config: rus_Cyrl-sna_Latn
name: MTEB FloresBitextMining (rus_Cyrl-sna_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 89.72332015810277
- type: f1
value: 87.30013645774514
- type: main_score
value: 87.30013645774514
- type: precision
value: 86.25329380764163
- type: recall
value: 89.72332015810277
task:
type: BitextMining
- dataset:
config: rus_Cyrl-tso_Latn
name: MTEB FloresBitextMining (rus_Cyrl-tso_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 84.38735177865613
- type: f1
value: 80.70424744337788
- type: main_score
value: 80.70424744337788
- type: precision
value: 79.18560606060606
- type: recall
value: 84.38735177865613
task:
type: BitextMining
- dataset:
config: rus_Cyrl-azj_Latn
name: MTEB FloresBitextMining (rus_Cyrl-azj_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.33201581027669
- type: f1
value: 96.56455862977602
- type: main_score
value: 96.56455862977602
- type: precision
value: 96.23682476943345
- type: recall
value: 97.33201581027669
task:
type: BitextMining
- dataset:
config: rus_Cyrl-dik_Latn
name: MTEB FloresBitextMining (rus_Cyrl-dik_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 46.047430830039524
- type: f1
value: 40.05513069495283
- type: main_score
value: 40.05513069495283
- type: precision
value: 38.072590197096126
- type: recall
value: 46.047430830039524
task:
type: BitextMining
- dataset:
config: rus_Cyrl-hau_Latn
name: MTEB FloresBitextMining (rus_Cyrl-hau_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 87.94466403162056
- type: f1
value: 84.76943346508563
- type: main_score
value: 84.76943346508563
- type: precision
value: 83.34486166007905
- type: recall
value: 87.94466403162056
task:
type: BitextMining
- dataset:
config: rus_Cyrl-kea_Latn
name: MTEB FloresBitextMining (rus_Cyrl-kea_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 89.42687747035573
- type: f1
value: 86.83803021747684
- type: main_score
value: 86.83803021747684
- type: precision
value: 85.78416149068323
- type: recall
value: 89.42687747035573
task:
type: BitextMining
- dataset:
config: rus_Cyrl-lus_Latn
name: MTEB FloresBitextMining (rus_Cyrl-lus_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 68.97233201581028
- type: f1
value: 64.05480726292745
- type: main_score
value: 64.05480726292745
- type: precision
value: 62.42670749487858
- type: recall
value: 68.97233201581028
task:
type: BitextMining
- dataset:
config: rus_Cyrl-pag_Latn
name: MTEB FloresBitextMining (rus_Cyrl-pag_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 78.75494071146245
- type: f1
value: 74.58573558401933
- type: main_score
value: 74.58573558401933
- type: precision
value: 73.05532028358115
- type: recall
value: 78.75494071146245
task:
type: BitextMining
- dataset:
config: rus_Cyrl-snd_Arab
name: MTEB FloresBitextMining (rus_Cyrl-snd_Arab)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 95.8498023715415
- type: f1
value: 94.56521739130434
- type: main_score
value: 94.56521739130434
- type: precision
value: 93.97233201581028
- type: recall
value: 95.8498023715415
task:
type: BitextMining
- dataset:
config: rus_Cyrl-tuk_Latn
name: MTEB FloresBitextMining (rus_Cyrl-tuk_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 68.08300395256917
- type: f1
value: 62.93565240205557
- type: main_score
value: 62.93565240205557
- type: precision
value: 61.191590257043934
- type: recall
value: 68.08300395256917
task:
type: BitextMining
- dataset:
config: rus_Cyrl-bak_Cyrl
name: MTEB FloresBitextMining (rus_Cyrl-bak_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 96.04743083003953
- type: f1
value: 94.86824769433464
- type: main_score
value: 94.86824769433464
- type: precision
value: 94.34288537549406
- type: recall
value: 96.04743083003953
task:
type: BitextMining
- dataset:
config: rus_Cyrl-dyu_Latn
name: MTEB FloresBitextMining (rus_Cyrl-dyu_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 37.45059288537549
- type: f1
value: 31.670482312800807
- type: main_score
value: 31.670482312800807
- type: precision
value: 29.99928568357422
- type: recall
value: 37.45059288537549
task:
type: BitextMining
- dataset:
config: rus_Cyrl-heb_Hebr
name: MTEB FloresBitextMining (rus_Cyrl-heb_Hebr)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.23320158102767
- type: f1
value: 96.38998682476942
- type: main_score
value: 96.38998682476942
- type: precision
value: 95.99802371541502
- type: recall
value: 97.23320158102767
task:
type: BitextMining
- dataset:
config: rus_Cyrl-khk_Cyrl
name: MTEB FloresBitextMining (rus_Cyrl-khk_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.41897233201581
- type: f1
value: 98.00724637681158
- type: main_score
value: 98.00724637681158
- type: precision
value: 97.82938076416336
- type: recall
value: 98.41897233201581
task:
type: BitextMining
- dataset:
config: rus_Cyrl-lvs_Latn
name: MTEB FloresBitextMining (rus_Cyrl-lvs_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.4308300395257
- type: f1
value: 96.61396574440053
- type: main_score
value: 96.61396574440053
- type: precision
value: 96.2203557312253
- type: recall
value: 97.4308300395257
task:
type: BitextMining
- dataset:
config: rus_Cyrl-pan_Guru
name: MTEB FloresBitextMining (rus_Cyrl-pan_Guru)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.30830039525692
- type: f1
value: 99.07773386034256
- type: main_score
value: 99.07773386034256
- type: precision
value: 98.96245059288538
- type: recall
value: 99.30830039525692
task:
type: BitextMining
- dataset:
config: rus_Cyrl-som_Latn
name: MTEB FloresBitextMining (rus_Cyrl-som_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 87.74703557312253
- type: f1
value: 84.52898550724638
- type: main_score
value: 84.52898550724638
- type: precision
value: 83.09288537549409
- type: recall
value: 87.74703557312253
task:
type: BitextMining
- dataset:
config: rus_Cyrl-tum_Latn
name: MTEB FloresBitextMining (rus_Cyrl-tum_Latn)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 87.15415019762845
- type: f1
value: 83.85069640504425
- type: main_score
value: 83.85069640504425
- type: precision
value: 82.43671183888576
- type: recall
value: 87.15415019762845
task:
type: BitextMining
- dataset:
config: taq_Latn-rus_Cyrl
name: MTEB FloresBitextMining (taq_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 28.55731225296443
- type: f1
value: 26.810726360049568
- type: main_score
value: 26.810726360049568
- type: precision
value: 26.260342858265577
- type: recall
value: 28.55731225296443
task:
type: BitextMining
- dataset:
config: war_Latn-rus_Cyrl
name: MTEB FloresBitextMining (war_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 94.86166007905138
- type: f1
value: 94.03147083483051
- type: main_score
value: 94.03147083483051
- type: precision
value: 93.70653606003322
- type: recall
value: 94.86166007905138
task:
type: BitextMining
- dataset:
config: arb_Arab-rus_Cyrl
name: MTEB FloresBitextMining (arb_Arab-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 96.34387351778656
- type: f1
value: 95.23056653491436
- type: main_score
value: 95.23056653491436
- type: precision
value: 94.70520421607378
- type: recall
value: 96.34387351778656
task:
type: BitextMining
- dataset:
config: bul_Cyrl-rus_Cyrl
name: MTEB FloresBitextMining (bul_Cyrl-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.90118577075098
- type: f1
value: 99.86824769433464
- type: main_score
value: 99.86824769433464
- type: precision
value: 99.85177865612648
- type: recall
value: 99.90118577075098
task:
type: BitextMining
- dataset:
config: fra_Latn-rus_Cyrl
name: MTEB FloresBitextMining (fra_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.2094861660079
- type: f1
value: 98.9459815546772
- type: main_score
value: 98.9459815546772
- type: precision
value: 98.81422924901186
- type: recall
value: 99.2094861660079
task:
type: BitextMining
- dataset:
config: jpn_Jpan-rus_Cyrl
name: MTEB FloresBitextMining (jpn_Jpan-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.3201581027668
- type: f1
value: 97.76021080368905
- type: main_score
value: 97.76021080368905
- type: precision
value: 97.48023715415019
- type: recall
value: 98.3201581027668
task:
type: BitextMining
- dataset:
config: lij_Latn-rus_Cyrl
name: MTEB FloresBitextMining (lij_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 83.49802371541502
- type: f1
value: 81.64800059239636
- type: main_score
value: 81.64800059239636
- type: precision
value: 80.9443055878478
- type: recall
value: 83.49802371541502
task:
type: BitextMining
- dataset:
config: mya_Mymr-rus_Cyrl
name: MTEB FloresBitextMining (mya_Mymr-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 90.21739130434783
- type: f1
value: 88.76776366313682
- type: main_score
value: 88.76776366313682
- type: precision
value: 88.18370446119435
- type: recall
value: 90.21739130434783
task:
type: BitextMining
- dataset:
config: sag_Latn-rus_Cyrl
name: MTEB FloresBitextMining (sag_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 41.699604743083
- type: f1
value: 39.53066322643847
- type: main_score
value: 39.53066322643847
- type: precision
value: 38.822876239229274
- type: recall
value: 41.699604743083
task:
type: BitextMining
- dataset:
config: taq_Tfng-rus_Cyrl
name: MTEB FloresBitextMining (taq_Tfng-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 10.67193675889328
- type: f1
value: 9.205744965817951
- type: main_score
value: 9.205744965817951
- type: precision
value: 8.85195219073817
- type: recall
value: 10.67193675889328
task:
type: BitextMining
- dataset:
config: wol_Latn-rus_Cyrl
name: MTEB FloresBitextMining (wol_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 63.537549407114625
- type: f1
value: 60.65190727391827
- type: main_score
value: 60.65190727391827
- type: precision
value: 59.61144833427442
- type: recall
value: 63.537549407114625
task:
type: BitextMining
- dataset:
config: arb_Latn-rus_Cyrl
name: MTEB FloresBitextMining (arb_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 13.142292490118576
- type: f1
value: 12.372910318176764
- type: main_score
value: 12.372910318176764
- type: precision
value: 12.197580895919188
- type: recall
value: 13.142292490118576
task:
type: BitextMining
- dataset:
config: cat_Latn-rus_Cyrl
name: MTEB FloresBitextMining (cat_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.01185770750988
- type: f1
value: 98.80599472990777
- type: main_score
value: 98.80599472990777
- type: precision
value: 98.72953133822698
- type: recall
value: 99.01185770750988
task:
type: BitextMining
- dataset:
config: fur_Latn-rus_Cyrl
name: MTEB FloresBitextMining (fur_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 81.02766798418972
- type: f1
value: 79.36184294084613
- type: main_score
value: 79.36184294084613
- type: precision
value: 78.69187826527705
- type: recall
value: 81.02766798418972
task:
type: BitextMining
- dataset:
config: kab_Latn-rus_Cyrl
name: MTEB FloresBitextMining (kab_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 34.387351778656125
- type: f1
value: 32.02306921576947
- type: main_score
value: 32.02306921576947
- type: precision
value: 31.246670347137467
- type: recall
value: 34.387351778656125
task:
type: BitextMining
- dataset:
config: lim_Latn-rus_Cyrl
name: MTEB FloresBitextMining (lim_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 78.26086956521739
- type: f1
value: 75.90239449214359
- type: main_score
value: 75.90239449214359
- type: precision
value: 75.02211430745493
- type: recall
value: 78.26086956521739
task:
type: BitextMining
- dataset:
config: nld_Latn-rus_Cyrl
name: MTEB FloresBitextMining (nld_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.2094861660079
- type: f1
value: 98.9459815546772
- type: main_score
value: 98.9459815546772
- type: precision
value: 98.81422924901186
- type: recall
value: 99.2094861660079
task:
type: BitextMining
- dataset:
config: san_Deva-rus_Cyrl
name: MTEB FloresBitextMining (san_Deva-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 87.94466403162056
- type: f1
value: 86.68928897189767
- type: main_score
value: 86.68928897189767
- type: precision
value: 86.23822997079216
- type: recall
value: 87.94466403162056
task:
type: BitextMining
- dataset:
config: tat_Cyrl-rus_Cyrl
name: MTEB FloresBitextMining (tat_Cyrl-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.03557312252964
- type: f1
value: 96.4167365353136
- type: main_score
value: 96.4167365353136
- type: precision
value: 96.16847826086958
- type: recall
value: 97.03557312252964
task:
type: BitextMining
- dataset:
config: xho_Latn-rus_Cyrl
name: MTEB FloresBitextMining (xho_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 86.95652173913044
- type: f1
value: 85.5506497283435
- type: main_score
value: 85.5506497283435
- type: precision
value: 84.95270479733395
- type: recall
value: 86.95652173913044
task:
type: BitextMining
- dataset:
config: ars_Arab-rus_Cyrl
name: MTEB FloresBitextMining (ars_Arab-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 96.6403162055336
- type: f1
value: 95.60935441370223
- type: main_score
value: 95.60935441370223
- type: precision
value: 95.13339920948617
- type: recall
value: 96.6403162055336
task:
type: BitextMining
- dataset:
config: ceb_Latn-rus_Cyrl
name: MTEB FloresBitextMining (ceb_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 95.7509881422925
- type: f1
value: 95.05209198303827
- type: main_score
value: 95.05209198303827
- type: precision
value: 94.77662283368805
- type: recall
value: 95.7509881422925
task:
type: BitextMining
- dataset:
config: fuv_Latn-rus_Cyrl
name: MTEB FloresBitextMining (fuv_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 45.25691699604743
- type: f1
value: 42.285666666742365
- type: main_score
value: 42.285666666742365
- type: precision
value: 41.21979853402283
- type: recall
value: 45.25691699604743
task:
type: BitextMining
- dataset:
config: kac_Latn-rus_Cyrl
name: MTEB FloresBitextMining (kac_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 34.683794466403164
- type: f1
value: 33.3235346229031
- type: main_score
value: 33.3235346229031
- type: precision
value: 32.94673924616852
- type: recall
value: 34.683794466403164
task:
type: BitextMining
- dataset:
config: lin_Latn-rus_Cyrl
name: MTEB FloresBitextMining (lin_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 86.85770750988142
- type: f1
value: 85.1867110799439
- type: main_score
value: 85.1867110799439
- type: precision
value: 84.53038212173273
- type: recall
value: 86.85770750988142
task:
type: BitextMining
- dataset:
config: nno_Latn-rus_Cyrl
name: MTEB FloresBitextMining (nno_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.4308300395257
- type: f1
value: 96.78383210991906
- type: main_score
value: 96.78383210991906
- type: precision
value: 96.51185770750989
- type: recall
value: 97.4308300395257
task:
type: BitextMining
- dataset:
config: sat_Olck-rus_Cyrl
name: MTEB FloresBitextMining (sat_Olck-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 1.185770750988142
- type: f1
value: 1.0279253129117258
- type: main_score
value: 1.0279253129117258
- type: precision
value: 1.0129746819135175
- type: recall
value: 1.185770750988142
task:
type: BitextMining
- dataset:
config: tel_Telu-rus_Cyrl
name: MTEB FloresBitextMining (tel_Telu-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.12252964426878
- type: f1
value: 97.61198945981555
- type: main_score
value: 97.61198945981555
- type: precision
value: 97.401185770751
- type: recall
value: 98.12252964426878
task:
type: BitextMining
- dataset:
config: ydd_Hebr-rus_Cyrl
name: MTEB FloresBitextMining (ydd_Hebr-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 75.8893280632411
- type: f1
value: 74.00244008018511
- type: main_score
value: 74.00244008018511
- type: precision
value: 73.25683020960382
- type: recall
value: 75.8893280632411
task:
type: BitextMining
- dataset:
config: ary_Arab-rus_Cyrl
name: MTEB FloresBitextMining (ary_Arab-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 86.56126482213439
- type: f1
value: 83.72796285839765
- type: main_score
value: 83.72796285839765
- type: precision
value: 82.65014273166447
- type: recall
value: 86.56126482213439
task:
type: BitextMining
- dataset:
config: ces_Latn-rus_Cyrl
name: MTEB FloresBitextMining (ces_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.60474308300395
- type: f1
value: 99.4729907773386
- type: main_score
value: 99.4729907773386
- type: precision
value: 99.40711462450594
- type: recall
value: 99.60474308300395
task:
type: BitextMining
- dataset:
config: gaz_Latn-rus_Cyrl
name: MTEB FloresBitextMining (gaz_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 42.58893280632411
- type: f1
value: 40.75832866805978
- type: main_score
value: 40.75832866805978
- type: precision
value: 40.14285046917723
- type: recall
value: 42.58893280632411
task:
type: BitextMining
- dataset:
config: kam_Latn-rus_Cyrl
name: MTEB FloresBitextMining (kam_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 45.25691699604743
- type: f1
value: 42.6975518029456
- type: main_score
value: 42.6975518029456
- type: precision
value: 41.87472710984596
- type: recall
value: 45.25691699604743
task:
type: BitextMining
- dataset:
config: lit_Latn-rus_Cyrl
name: MTEB FloresBitextMining (lit_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.33201581027669
- type: f1
value: 96.62384716732542
- type: main_score
value: 96.62384716732542
- type: precision
value: 96.3175230566535
- type: recall
value: 97.33201581027669
task:
type: BitextMining
- dataset:
config: nob_Latn-rus_Cyrl
name: MTEB FloresBitextMining (nob_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.71541501976284
- type: f1
value: 98.30368906455863
- type: main_score
value: 98.30368906455863
- type: precision
value: 98.10606060606061
- type: recall
value: 98.71541501976284
task:
type: BitextMining
- dataset:
config: scn_Latn-rus_Cyrl
name: MTEB FloresBitextMining (scn_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 70.45454545454545
- type: f1
value: 68.62561022640075
- type: main_score
value: 68.62561022640075
- type: precision
value: 67.95229103411222
- type: recall
value: 70.45454545454545
task:
type: BitextMining
- dataset:
config: tgk_Cyrl-rus_Cyrl
name: MTEB FloresBitextMining (tgk_Cyrl-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 92.4901185770751
- type: f1
value: 91.58514492753623
- type: main_score
value: 91.58514492753623
- type: precision
value: 91.24759298672342
- type: recall
value: 92.4901185770751
task:
type: BitextMining
- dataset:
config: yor_Latn-rus_Cyrl
name: MTEB FloresBitextMining (yor_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 67.98418972332016
- type: f1
value: 64.72874247330768
- type: main_score
value: 64.72874247330768
- type: precision
value: 63.450823399938685
- type: recall
value: 67.98418972332016
task:
type: BitextMining
- dataset:
config: arz_Arab-rus_Cyrl
name: MTEB FloresBitextMining (arz_Arab-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 94.56521739130434
- type: f1
value: 93.07971014492755
- type: main_score
value: 93.07971014492755
- type: precision
value: 92.42753623188406
- type: recall
value: 94.56521739130434
task:
type: BitextMining
- dataset:
config: cjk_Latn-rus_Cyrl
name: MTEB FloresBitextMining (cjk_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 38.63636363636363
- type: f1
value: 36.25747140862938
- type: main_score
value: 36.25747140862938
- type: precision
value: 35.49101355074723
- type: recall
value: 38.63636363636363
task:
type: BitextMining
- dataset:
config: gla_Latn-rus_Cyrl
name: MTEB FloresBitextMining (gla_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 69.26877470355731
- type: f1
value: 66.11797423328613
- type: main_score
value: 66.11797423328613
- type: precision
value: 64.89369649409694
- type: recall
value: 69.26877470355731
task:
type: BitextMining
- dataset:
config: kan_Knda-rus_Cyrl
name: MTEB FloresBitextMining (kan_Knda-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.02371541501977
- type: f1
value: 97.51505740636176
- type: main_score
value: 97.51505740636176
- type: precision
value: 97.30731225296442
- type: recall
value: 98.02371541501977
task:
type: BitextMining
- dataset:
config: lmo_Latn-rus_Cyrl
name: MTEB FloresBitextMining (lmo_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 73.3201581027668
- type: f1
value: 71.06371608677273
- type: main_score
value: 71.06371608677273
- type: precision
value: 70.26320288266223
- type: recall
value: 73.3201581027668
task:
type: BitextMining
- dataset:
config: npi_Deva-rus_Cyrl
name: MTEB FloresBitextMining (npi_Deva-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.82608695652173
- type: f1
value: 97.36645107198466
- type: main_score
value: 97.36645107198466
- type: precision
value: 97.1772068511199
- type: recall
value: 97.82608695652173
task:
type: BitextMining
- dataset:
config: shn_Mymr-rus_Cyrl
name: MTEB FloresBitextMining (shn_Mymr-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 39.426877470355734
- type: f1
value: 37.16728785513024
- type: main_score
value: 37.16728785513024
- type: precision
value: 36.56918548278505
- type: recall
value: 39.426877470355734
task:
type: BitextMining
- dataset:
config: tgl_Latn-rus_Cyrl
name: MTEB FloresBitextMining (tgl_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.92490118577075
- type: f1
value: 97.6378693769998
- type: main_score
value: 97.6378693769998
- type: precision
value: 97.55371440154047
- type: recall
value: 97.92490118577075
task:
type: BitextMining
- dataset:
config: yue_Hant-rus_Cyrl
name: MTEB FloresBitextMining (yue_Hant-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.92490118577075
- type: f1
value: 97.3833051006964
- type: main_score
value: 97.3833051006964
- type: precision
value: 97.1590909090909
- type: recall
value: 97.92490118577075
task:
type: BitextMining
- dataset:
config: asm_Beng-rus_Cyrl
name: MTEB FloresBitextMining (asm_Beng-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 92.78656126482213
- type: f1
value: 91.76917395296842
- type: main_score
value: 91.76917395296842
- type: precision
value: 91.38292866553736
- type: recall
value: 92.78656126482213
task:
type: BitextMining
- dataset:
config: ckb_Arab-rus_Cyrl
name: MTEB FloresBitextMining (ckb_Arab-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 80.8300395256917
- type: f1
value: 79.17664345468799
- type: main_score
value: 79.17664345468799
- type: precision
value: 78.5622171683459
- type: recall
value: 80.8300395256917
task:
type: BitextMining
- dataset:
config: gle_Latn-rus_Cyrl
name: MTEB FloresBitextMining (gle_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 85.86956521739131
- type: f1
value: 84.45408265372492
- type: main_score
value: 84.45408265372492
- type: precision
value: 83.8774340026703
- type: recall
value: 85.86956521739131
task:
type: BitextMining
- dataset:
config: kas_Arab-rus_Cyrl
name: MTEB FloresBitextMining (kas_Arab-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 76.28458498023716
- type: f1
value: 74.11216313578267
- type: main_score
value: 74.11216313578267
- type: precision
value: 73.2491277759584
- type: recall
value: 76.28458498023716
task:
type: BitextMining
- dataset:
config: ltg_Latn-rus_Cyrl
name: MTEB FloresBitextMining (ltg_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 71.14624505928853
- type: f1
value: 68.69245357723618
- type: main_score
value: 68.69245357723618
- type: precision
value: 67.8135329666459
- type: recall
value: 71.14624505928853
task:
type: BitextMining
- dataset:
config: nso_Latn-rus_Cyrl
name: MTEB FloresBitextMining (nso_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 87.64822134387352
- type: f1
value: 85.98419219986725
- type: main_score
value: 85.98419219986725
- type: precision
value: 85.32513873917036
- type: recall
value: 87.64822134387352
task:
type: BitextMining
- dataset:
config: sin_Sinh-rus_Cyrl
name: MTEB FloresBitextMining (sin_Sinh-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.62845849802372
- type: f1
value: 97.10144927536231
- type: main_score
value: 97.10144927536231
- type: precision
value: 96.87986585219788
- type: recall
value: 97.62845849802372
task:
type: BitextMining
- dataset:
config: tha_Thai-rus_Cyrl
name: MTEB FloresBitextMining (tha_Thai-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.71541501976284
- type: f1
value: 98.28722002635045
- type: main_score
value: 98.28722002635045
- type: precision
value: 98.07312252964427
- type: recall
value: 98.71541501976284
task:
type: BitextMining
- dataset:
config: zho_Hans-rus_Cyrl
name: MTEB FloresBitextMining (zho_Hans-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.01185770750988
- type: f1
value: 98.68247694334651
- type: main_score
value: 98.68247694334651
- type: precision
value: 98.51778656126481
- type: recall
value: 99.01185770750988
task:
type: BitextMining
- dataset:
config: ast_Latn-rus_Cyrl
name: MTEB FloresBitextMining (ast_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 95.65217391304348
- type: f1
value: 94.90649683857505
- type: main_score
value: 94.90649683857505
- type: precision
value: 94.61352657004831
- type: recall
value: 95.65217391304348
task:
type: BitextMining
- dataset:
config: crh_Latn-rus_Cyrl
name: MTEB FloresBitextMining (crh_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 93.08300395256917
- type: f1
value: 92.20988998886428
- type: main_score
value: 92.20988998886428
- type: precision
value: 91.85631013694254
- type: recall
value: 93.08300395256917
task:
type: BitextMining
- dataset:
config: glg_Latn-rus_Cyrl
name: MTEB FloresBitextMining (glg_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 95.55335968379447
- type: f1
value: 95.18006148440931
- type: main_score
value: 95.18006148440931
- type: precision
value: 95.06540560888386
- type: recall
value: 95.55335968379447
task:
type: BitextMining
- dataset:
config: kas_Deva-rus_Cyrl
name: MTEB FloresBitextMining (kas_Deva-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 55.03952569169961
- type: f1
value: 52.19871938895554
- type: main_score
value: 52.19871938895554
- type: precision
value: 51.17660971469557
- type: recall
value: 55.03952569169961
task:
type: BitextMining
- dataset:
config: ltz_Latn-rus_Cyrl
name: MTEB FloresBitextMining (ltz_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 87.64822134387352
- type: f1
value: 86.64179841897234
- type: main_score
value: 86.64179841897234
- type: precision
value: 86.30023235431587
- type: recall
value: 87.64822134387352
task:
type: BitextMining
- dataset:
config: nus_Latn-rus_Cyrl
name: MTEB FloresBitextMining (nus_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 27.4703557312253
- type: f1
value: 25.703014277858088
- type: main_score
value: 25.703014277858088
- type: precision
value: 25.194105476917315
- type: recall
value: 27.4703557312253
task:
type: BitextMining
- dataset:
config: slk_Latn-rus_Cyrl
name: MTEB FloresBitextMining (slk_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.30830039525692
- type: f1
value: 99.1106719367589
- type: main_score
value: 99.1106719367589
- type: precision
value: 99.02832674571805
- type: recall
value: 99.30830039525692
task:
type: BitextMining
- dataset:
config: tir_Ethi-rus_Cyrl
name: MTEB FloresBitextMining (tir_Ethi-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 80.73122529644269
- type: f1
value: 78.66903754775608
- type: main_score
value: 78.66903754775608
- type: precision
value: 77.86431694163612
- type: recall
value: 80.73122529644269
task:
type: BitextMining
- dataset:
config: zho_Hant-rus_Cyrl
name: MTEB FloresBitextMining (zho_Hant-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.22134387351778
- type: f1
value: 97.66798418972333
- type: main_score
value: 97.66798418972333
- type: precision
value: 97.40612648221344
- type: recall
value: 98.22134387351778
task:
type: BitextMining
- dataset:
config: awa_Deva-rus_Cyrl
name: MTEB FloresBitextMining (awa_Deva-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.5296442687747
- type: f1
value: 96.94224857268335
- type: main_score
value: 96.94224857268335
- type: precision
value: 96.68560606060606
- type: recall
value: 97.5296442687747
task:
type: BitextMining
- dataset:
config: cym_Latn-rus_Cyrl
name: MTEB FloresBitextMining (cym_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 92.68774703557312
- type: f1
value: 91.69854302097961
- type: main_score
value: 91.69854302097961
- type: precision
value: 91.31236846157795
- type: recall
value: 92.68774703557312
task:
type: BitextMining
- dataset:
config: grn_Latn-rus_Cyrl
name: MTEB FloresBitextMining (grn_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 64.13043478260869
- type: f1
value: 61.850586118740004
- type: main_score
value: 61.850586118740004
- type: precision
value: 61.0049495186209
- type: recall
value: 64.13043478260869
task:
type: BitextMining
- dataset:
config: kat_Geor-rus_Cyrl
name: MTEB FloresBitextMining (kat_Geor-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.02371541501977
- type: f1
value: 97.59881422924902
- type: main_score
value: 97.59881422924902
- type: precision
value: 97.42534036012296
- type: recall
value: 98.02371541501977
task:
type: BitextMining
- dataset:
config: lua_Latn-rus_Cyrl
name: MTEB FloresBitextMining (lua_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 63.63636363636363
- type: f1
value: 60.9709122526128
- type: main_score
value: 60.9709122526128
- type: precision
value: 60.03915902282226
- type: recall
value: 63.63636363636363
task:
type: BitextMining
- dataset:
config: nya_Latn-rus_Cyrl
name: MTEB FloresBitextMining (nya_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 89.2292490118577
- type: f1
value: 87.59723824473149
- type: main_score
value: 87.59723824473149
- type: precision
value: 86.90172707867349
- type: recall
value: 89.2292490118577
task:
type: BitextMining
- dataset:
config: slv_Latn-rus_Cyrl
name: MTEB FloresBitextMining (slv_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.01185770750988
- type: f1
value: 98.74835309617917
- type: main_score
value: 98.74835309617917
- type: precision
value: 98.63636363636364
- type: recall
value: 99.01185770750988
task:
type: BitextMining
- dataset:
config: tpi_Latn-rus_Cyrl
name: MTEB FloresBitextMining (tpi_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 77.37154150197628
- type: f1
value: 75.44251611276084
- type: main_score
value: 75.44251611276084
- type: precision
value: 74.78103665109595
- type: recall
value: 77.37154150197628
task:
type: BitextMining
- dataset:
config: zsm_Latn-rus_Cyrl
name: MTEB FloresBitextMining (zsm_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.2094861660079
- type: f1
value: 98.96245059288538
- type: main_score
value: 98.96245059288538
- type: precision
value: 98.8471673254282
- type: recall
value: 99.2094861660079
task:
type: BitextMining
- dataset:
config: ayr_Latn-rus_Cyrl
name: MTEB FloresBitextMining (ayr_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 27.766798418972332
- type: f1
value: 26.439103195281312
- type: main_score
value: 26.439103195281312
- type: precision
value: 26.052655604573964
- type: recall
value: 27.766798418972332
task:
type: BitextMining
- dataset:
config: dan_Latn-rus_Cyrl
name: MTEB FloresBitextMining (dan_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.30830039525692
- type: f1
value: 99.07773386034255
- type: main_score
value: 99.07773386034255
- type: precision
value: 98.96245059288538
- type: recall
value: 99.30830039525692
task:
type: BitextMining
- dataset:
config: guj_Gujr-rus_Cyrl
name: MTEB FloresBitextMining (guj_Gujr-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.82608695652173
- type: f1
value: 97.26449275362317
- type: main_score
value: 97.26449275362317
- type: precision
value: 97.02498588368154
- type: recall
value: 97.82608695652173
task:
type: BitextMining
- dataset:
config: kaz_Cyrl-rus_Cyrl
name: MTEB FloresBitextMining (kaz_Cyrl-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.5296442687747
- type: f1
value: 97.03557312252964
- type: main_score
value: 97.03557312252964
- type: precision
value: 96.85022158342316
- type: recall
value: 97.5296442687747
task:
type: BitextMining
- dataset:
config: lug_Latn-rus_Cyrl
name: MTEB FloresBitextMining (lug_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 68.57707509881423
- type: f1
value: 65.93361605820395
- type: main_score
value: 65.93361605820395
- type: precision
value: 64.90348248593789
- type: recall
value: 68.57707509881423
task:
type: BitextMining
- dataset:
config: oci_Latn-rus_Cyrl
name: MTEB FloresBitextMining (oci_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 86.26482213438736
- type: f1
value: 85.33176417155623
- type: main_score
value: 85.33176417155623
- type: precision
value: 85.00208833384637
- type: recall
value: 86.26482213438736
task:
type: BitextMining
- dataset:
config: smo_Latn-rus_Cyrl
name: MTEB FloresBitextMining (smo_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 77.96442687747036
- type: f1
value: 75.70960450188885
- type: main_score
value: 75.70960450188885
- type: precision
value: 74.8312632736777
- type: recall
value: 77.96442687747036
task:
type: BitextMining
- dataset:
config: tsn_Latn-rus_Cyrl
name: MTEB FloresBitextMining (tsn_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 84.38735177865613
- type: f1
value: 82.13656376349225
- type: main_score
value: 82.13656376349225
- type: precision
value: 81.16794543904518
- type: recall
value: 84.38735177865613
task:
type: BitextMining
- dataset:
config: zul_Latn-rus_Cyrl
name: MTEB FloresBitextMining (zul_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 90.21739130434783
- type: f1
value: 88.77570602050753
- type: main_score
value: 88.77570602050753
- type: precision
value: 88.15978104021582
- type: recall
value: 90.21739130434783
task:
type: BitextMining
- dataset:
config: azb_Arab-rus_Cyrl
name: MTEB FloresBitextMining (azb_Arab-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 65.71146245059289
- type: f1
value: 64.18825390221271
- type: main_score
value: 64.18825390221271
- type: precision
value: 63.66811154793568
- type: recall
value: 65.71146245059289
task:
type: BitextMining
- dataset:
config: deu_Latn-rus_Cyrl
name: MTEB FloresBitextMining (deu_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 99.70355731225297
- type: f1
value: 99.60474308300395
- type: main_score
value: 99.60474308300395
- type: precision
value: 99.55533596837944
- type: recall
value: 99.70355731225297
task:
type: BitextMining
- dataset:
config: hat_Latn-rus_Cyrl
name: MTEB FloresBitextMining (hat_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 86.7588932806324
- type: f1
value: 85.86738623695146
- type: main_score
value: 85.86738623695146
- type: precision
value: 85.55235467420822
- type: recall
value: 86.7588932806324
task:
type: BitextMining
- dataset:
config: kbp_Latn-rus_Cyrl
name: MTEB FloresBitextMining (kbp_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 34.88142292490119
- type: f1
value: 32.16511669463015
- type: main_score
value: 32.16511669463015
- type: precision
value: 31.432098549546318
- type: recall
value: 34.88142292490119
task:
type: BitextMining
- dataset:
config: luo_Latn-rus_Cyrl
name: MTEB FloresBitextMining (luo_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 52.27272727272727
- type: f1
value: 49.60489626836975
- type: main_score
value: 49.60489626836975
- type: precision
value: 48.69639631803339
- type: recall
value: 52.27272727272727
task:
type: BitextMining
- dataset:
config: ory_Orya-rus_Cyrl
name: MTEB FloresBitextMining (ory_Orya-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.82608695652173
- type: f1
value: 97.27437417654808
- type: main_score
value: 97.27437417654808
- type: precision
value: 97.04968944099377
- type: recall
value: 97.82608695652173
task:
type: BitextMining
- dataset:
config: sna_Latn-rus_Cyrl
name: MTEB FloresBitextMining (sna_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 85.37549407114624
- type: f1
value: 83.09911316305177
- type: main_score
value: 83.09911316305177
- type: precision
value: 82.1284950958864
- type: recall
value: 85.37549407114624
task:
type: BitextMining
- dataset:
config: tso_Latn-rus_Cyrl
name: MTEB FloresBitextMining (tso_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 82.90513833992095
- type: f1
value: 80.28290385503824
- type: main_score
value: 80.28290385503824
- type: precision
value: 79.23672543237761
- type: recall
value: 82.90513833992095
task:
type: BitextMining
- dataset:
config: azj_Latn-rus_Cyrl
name: MTEB FloresBitextMining (azj_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.02371541501977
- type: f1
value: 97.49200075287031
- type: main_score
value: 97.49200075287031
- type: precision
value: 97.266139657444
- type: recall
value: 98.02371541501977
task:
type: BitextMining
- dataset:
config: dik_Latn-rus_Cyrl
name: MTEB FloresBitextMining (dik_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 38.43873517786561
- type: f1
value: 35.78152442955223
- type: main_score
value: 35.78152442955223
- type: precision
value: 34.82424325078237
- type: recall
value: 38.43873517786561
task:
type: BitextMining
- dataset:
config: hau_Latn-rus_Cyrl
name: MTEB FloresBitextMining (hau_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 81.42292490118577
- type: f1
value: 79.24612283124593
- type: main_score
value: 79.24612283124593
- type: precision
value: 78.34736070751448
- type: recall
value: 81.42292490118577
task:
type: BitextMining
- dataset:
config: kea_Latn-rus_Cyrl
name: MTEB FloresBitextMining (kea_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 81.62055335968378
- type: f1
value: 80.47015182884748
- type: main_score
value: 80.47015182884748
- type: precision
value: 80.02671028885862
- type: recall
value: 81.62055335968378
task:
type: BitextMining
- dataset:
config: lus_Latn-rus_Cyrl
name: MTEB FloresBitextMining (lus_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 62.74703557312253
- type: f1
value: 60.53900079111122
- type: main_score
value: 60.53900079111122
- type: precision
value: 59.80024202850289
- type: recall
value: 62.74703557312253
task:
type: BitextMining
- dataset:
config: pag_Latn-rus_Cyrl
name: MTEB FloresBitextMining (pag_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 74.01185770750988
- type: f1
value: 72.57280648279529
- type: main_score
value: 72.57280648279529
- type: precision
value: 71.99952968456789
- type: recall
value: 74.01185770750988
task:
type: BitextMining
- dataset:
config: snd_Arab-rus_Cyrl
name: MTEB FloresBitextMining (snd_Arab-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 91.30434782608695
- type: f1
value: 90.24653499445358
- type: main_score
value: 90.24653499445358
- type: precision
value: 89.83134068200232
- type: recall
value: 91.30434782608695
task:
type: BitextMining
- dataset:
config: tuk_Latn-rus_Cyrl
name: MTEB FloresBitextMining (tuk_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 47.62845849802372
- type: f1
value: 45.812928836644254
- type: main_score
value: 45.812928836644254
- type: precision
value: 45.23713833170355
- type: recall
value: 47.62845849802372
task:
type: BitextMining
- dataset:
config: bak_Cyrl-rus_Cyrl
name: MTEB FloresBitextMining (bak_Cyrl-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 95.8498023715415
- type: f1
value: 95.18904459615922
- type: main_score
value: 95.18904459615922
- type: precision
value: 94.92812441182006
- type: recall
value: 95.8498023715415
task:
type: BitextMining
- dataset:
config: dyu_Latn-rus_Cyrl
name: MTEB FloresBitextMining (dyu_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 29.64426877470356
- type: f1
value: 27.287335193938166
- type: main_score
value: 27.287335193938166
- type: precision
value: 26.583996026587492
- type: recall
value: 29.64426877470356
task:
type: BitextMining
- dataset:
config: heb_Hebr-rus_Cyrl
name: MTEB FloresBitextMining (heb_Hebr-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 98.91304347826086
- type: f1
value: 98.55072463768116
- type: main_score
value: 98.55072463768116
- type: precision
value: 98.36956521739131
- type: recall
value: 98.91304347826086
task:
type: BitextMining
- dataset:
config: khk_Cyrl-rus_Cyrl
name: MTEB FloresBitextMining (khk_Cyrl-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 95.15810276679841
- type: f1
value: 94.44009547764487
- type: main_score
value: 94.44009547764487
- type: precision
value: 94.16579797014579
- type: recall
value: 95.15810276679841
task:
type: BitextMining
- dataset:
config: lvs_Latn-rus_Cyrl
name: MTEB FloresBitextMining (lvs_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.92490118577075
- type: f1
value: 97.51467241585817
- type: main_score
value: 97.51467241585817
- type: precision
value: 97.36166007905138
- type: recall
value: 97.92490118577075
task:
type: BitextMining
- dataset:
config: pan_Guru-rus_Cyrl
name: MTEB FloresBitextMining (pan_Guru-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 97.92490118577075
- type: f1
value: 97.42918313570486
- type: main_score
value: 97.42918313570486
- type: precision
value: 97.22261434217955
- type: recall
value: 97.92490118577075
task:
type: BitextMining
- dataset:
config: som_Latn-rus_Cyrl
name: MTEB FloresBitextMining (som_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 75.69169960474308
- type: f1
value: 73.7211667065916
- type: main_score
value: 73.7211667065916
- type: precision
value: 72.95842401892384
- type: recall
value: 75.69169960474308
task:
type: BitextMining
- dataset:
config: tum_Latn-rus_Cyrl
name: MTEB FloresBitextMining (tum_Latn-rus_Cyrl)
revision: e6b647fcb6299a2f686f742f4d4c023e553ea67e
split: devtest
type: mteb/flores
metrics:
- type: accuracy
value: 85.67193675889328
- type: f1
value: 82.9296066252588
- type: main_score
value: 82.9296066252588
- type: precision
value: 81.77330225447936
- type: recall
value: 85.67193675889328
task:
type: BitextMining
- dataset:
config: default
name: MTEB GeoreviewClassification (default)
revision: 3765c0d1de6b7d264bc459433c45e5a75513839c
split: test
type: ai-forever/georeview-classification
metrics:
- type: accuracy
value: 44.6630859375
- type: f1
value: 42.607425073610536
- type: f1_weighted
value: 42.60639474586065
- type: main_score
value: 44.6630859375
task:
type: Classification
- dataset:
config: default
name: MTEB GeoreviewClusteringP2P (default)
revision: 97a313c8fc85b47f13f33e7e9a95c1ad888c7fec
split: test
type: ai-forever/georeview-clustering-p2p
metrics:
- type: main_score
value: 58.15951247070825
- type: v_measure
value: 58.15951247070825
- type: v_measure_std
value: 0.6739615788288809
task:
type: Clustering
- dataset:
config: default
name: MTEB HeadlineClassification (default)
revision: 2fe05ee6b5832cda29f2ef7aaad7b7fe6a3609eb
split: test
type: ai-forever/headline-classification
metrics:
- type: accuracy
value: 73.935546875
- type: f1
value: 73.8654872186846
- type: f1_weighted
value: 73.86733122685095
- type: main_score
value: 73.935546875
task:
type: Classification
- dataset:
config: default
name: MTEB InappropriatenessClassification (default)
revision: 601651fdc45ef243751676e62dd7a19f491c0285
split: test
type: ai-forever/inappropriateness-classification
metrics:
- type: accuracy
value: 59.16015624999999
- type: ap
value: 55.52276605836938
- type: ap_weighted
value: 55.52276605836938
- type: f1
value: 58.614248199637956
- type: f1_weighted
value: 58.614248199637956
- type: main_score
value: 59.16015624999999
task:
type: Classification
- dataset:
config: default
name: MTEB KinopoiskClassification (default)
revision: 5911f26666ac11af46cb9c6849d0dc80a378af24
split: test
type: ai-forever/kinopoisk-sentiment-classification
metrics:
- type: accuracy
value: 49.959999999999994
- type: f1
value: 48.4900332316098
- type: f1_weighted
value: 48.4900332316098
- type: main_score
value: 49.959999999999994
task:
type: Classification
- dataset:
config: default
name: MTEB LanguageClassification (default)
revision: aa56583bf2bc52b0565770607d6fc3faebecf9e2
split: test
type: papluca/language-identification
metrics:
- type: accuracy
value: 71.005859375
- type: f1
value: 69.63481100303348
- type: f1_weighted
value: 69.64640413409529
- type: main_score
value: 71.005859375
task:
type: Classification
- dataset:
config: ru
name: MTEB MLSUMClusteringP2P (ru)
revision: b5d54f8f3b61ae17845046286940f03c6bc79bc7
split: test
type: reciTAL/mlsum
metrics:
- type: main_score
value: 42.11280087032343
- type: v_measure
value: 42.11280087032343
- type: v_measure_std
value: 6.7619971723605135
task:
type: Clustering
- dataset:
config: ru
name: MTEB MLSUMClusteringP2P.v2 (ru)
revision: b5d54f8f3b61ae17845046286940f03c6bc79bc7
split: test
type: reciTAL/mlsum
metrics:
- type: main_score
value: 43.00112546945811
- type: v_measure
value: 43.00112546945811
- type: v_measure_std
value: 1.4740560414835675
task:
type: Clustering
- dataset:
config: ru
name: MTEB MLSUMClusteringS2S (ru)
revision: b5d54f8f3b61ae17845046286940f03c6bc79bc7
split: test
type: reciTAL/mlsum
metrics:
- type: main_score
value: 39.81446080575161
- type: v_measure
value: 39.81446080575161
- type: v_measure_std
value: 7.125661320308298
task:
type: Clustering
- dataset:
config: ru
name: MTEB MLSUMClusteringS2S.v2 (ru)
revision: b5d54f8f3b61ae17845046286940f03c6bc79bc7
split: test
type: reciTAL/mlsum
metrics:
- type: main_score
value: 39.29659668980239
- type: v_measure
value: 39.29659668980239
- type: v_measure_std
value: 2.6570502923023094
task:
type: Clustering
- dataset:
config: ru
name: MTEB MultiLongDocRetrieval (ru)
revision: d67138e705d963e346253a80e59676ddb418810a
split: dev
type: Shitao/MLDR
metrics:
- type: main_score
value: 38.671
- type: map_at_1
value: 30.0
- type: map_at_10
value: 36.123
- type: map_at_100
value: 36.754999999999995
- type: map_at_1000
value: 36.806
- type: map_at_20
value: 36.464
- type: map_at_3
value: 35.25
- type: map_at_5
value: 35.8
- type: mrr_at_1
value: 30.0
- type: mrr_at_10
value: 36.122817460317464
- type: mrr_at_100
value: 36.75467016625293
- type: mrr_at_1000
value: 36.80612724920882
- type: mrr_at_20
value: 36.46359681984682
- type: mrr_at_3
value: 35.25
- type: mrr_at_5
value: 35.800000000000004
- type: nauc_map_at_1000_diff1
value: 55.61987610843598
- type: nauc_map_at_1000_max
value: 52.506795017152186
- type: nauc_map_at_1000_std
value: 2.95487192066911
- type: nauc_map_at_100_diff1
value: 55.598419532054734
- type: nauc_map_at_100_max
value: 52.48192017040307
- type: nauc_map_at_100_std
value: 2.930120252521189
- type: nauc_map_at_10_diff1
value: 56.02309155375198
- type: nauc_map_at_10_max
value: 52.739573233234424
- type: nauc_map_at_10_std
value: 2.4073432421641545
- type: nauc_map_at_1_diff1
value: 52.57059856776112
- type: nauc_map_at_1_max
value: 50.55668152952304
- type: nauc_map_at_1_std
value: 1.6572084853398048
- type: nauc_map_at_20_diff1
value: 55.75769029917031
- type: nauc_map_at_20_max
value: 52.53663737242853
- type: nauc_map_at_20_std
value: 2.8489192879814
- type: nauc_map_at_3_diff1
value: 56.90294128342709
- type: nauc_map_at_3_max
value: 53.10608389782041
- type: nauc_map_at_3_std
value: 1.4909731657889491
- type: nauc_map_at_5_diff1
value: 56.1258315436073
- type: nauc_map_at_5_max
value: 52.398078357541564
- type: nauc_map_at_5_std
value: 1.8256862015101467
- type: nauc_mrr_at_1000_diff1
value: 55.61987610843598
- type: nauc_mrr_at_1000_max
value: 52.506795017152186
- type: nauc_mrr_at_1000_std
value: 2.95487192066911
- type: nauc_mrr_at_100_diff1
value: 55.598419532054734
- type: nauc_mrr_at_100_max
value: 52.48192017040307
- type: nauc_mrr_at_100_std
value: 2.930120252521189
- type: nauc_mrr_at_10_diff1
value: 56.02309155375198
- type: nauc_mrr_at_10_max
value: 52.739573233234424
- type: nauc_mrr_at_10_std
value: 2.4073432421641545
- type: nauc_mrr_at_1_diff1
value: 52.57059856776112
- type: nauc_mrr_at_1_max
value: 50.55668152952304
- type: nauc_mrr_at_1_std
value: 1.6572084853398048
- type: nauc_mrr_at_20_diff1
value: 55.75769029917031
- type: nauc_mrr_at_20_max
value: 52.53663737242853
- type: nauc_mrr_at_20_std
value: 2.8489192879814
- type: nauc_mrr_at_3_diff1
value: 56.90294128342709
- type: nauc_mrr_at_3_max
value: 53.10608389782041
- type: nauc_mrr_at_3_std
value: 1.4909731657889491
- type: nauc_mrr_at_5_diff1
value: 56.1258315436073
- type: nauc_mrr_at_5_max
value: 52.398078357541564
- type: nauc_mrr_at_5_std
value: 1.8256862015101467
- type: nauc_ndcg_at_1000_diff1
value: 55.30733548408918
- type: nauc_ndcg_at_1000_max
value: 53.51143366189318
- type: nauc_ndcg_at_1000_std
value: 7.133789405525702
- type: nauc_ndcg_at_100_diff1
value: 54.32209039488095
- type: nauc_ndcg_at_100_max
value: 52.67499334461009
- type: nauc_ndcg_at_100_std
value: 6.878823275077807
- type: nauc_ndcg_at_10_diff1
value: 56.266780806997716
- type: nauc_ndcg_at_10_max
value: 53.52837255793743
- type: nauc_ndcg_at_10_std
value: 3.756832592964262
- type: nauc_ndcg_at_1_diff1
value: 52.57059856776112
- type: nauc_ndcg_at_1_max
value: 50.55668152952304
- type: nauc_ndcg_at_1_std
value: 1.6572084853398048
- type: nauc_ndcg_at_20_diff1
value: 55.39255420432796
- type: nauc_ndcg_at_20_max
value: 52.946114684072235
- type: nauc_ndcg_at_20_std
value: 5.414933414031693
- type: nauc_ndcg_at_3_diff1
value: 57.92826624996289
- type: nauc_ndcg_at_3_max
value: 53.89907760306972
- type: nauc_ndcg_at_3_std
value: 1.6661401245309218
- type: nauc_ndcg_at_5_diff1
value: 56.47508936029308
- type: nauc_ndcg_at_5_max
value: 52.66800998045517
- type: nauc_ndcg_at_5_std
value: 2.4127296184140423
- type: nauc_precision_at_1000_diff1
value: 57.25924020238401
- type: nauc_precision_at_1000_max
value: 65.1132590931922
- type: nauc_precision_at_1000_std
value: 40.60788709618145
- type: nauc_precision_at_100_diff1
value: 46.49620002554606
- type: nauc_precision_at_100_max
value: 53.02960148167071
- type: nauc_precision_at_100_std
value: 28.206028867032863
- type: nauc_precision_at_10_diff1
value: 56.562744749606765
- type: nauc_precision_at_10_max
value: 56.00594967783547
- type: nauc_precision_at_10_std
value: 8.368379831645163
- type: nauc_precision_at_1_diff1
value: 52.57059856776112
- type: nauc_precision_at_1_max
value: 50.55668152952304
- type: nauc_precision_at_1_std
value: 1.6572084853398048
- type: nauc_precision_at_20_diff1
value: 53.25915754614111
- type: nauc_precision_at_20_max
value: 54.03255118937036
- type: nauc_precision_at_20_std
value: 15.161611674272718
- type: nauc_precision_at_3_diff1
value: 60.726785748943854
- type: nauc_precision_at_3_max
value: 56.139896875869354
- type: nauc_precision_at_3_std
value: 2.2306901035769893
- type: nauc_precision_at_5_diff1
value: 57.1201127525187
- type: nauc_precision_at_5_max
value: 53.28665761862506
- type: nauc_precision_at_5_std
value: 4.358720050112237
- type: nauc_recall_at_1000_diff1
value: 57.259240202383964
- type: nauc_recall_at_1000_max
value: 65.11325909319218
- type: nauc_recall_at_1000_std
value: 40.60788709618142
- type: nauc_recall_at_100_diff1
value: 46.49620002554603
- type: nauc_recall_at_100_max
value: 53.02960148167071
- type: nauc_recall_at_100_std
value: 28.206028867032835
- type: nauc_recall_at_10_diff1
value: 56.562744749606765
- type: nauc_recall_at_10_max
value: 56.00594967783549
- type: nauc_recall_at_10_std
value: 8.368379831645147
- type: nauc_recall_at_1_diff1
value: 52.57059856776112
- type: nauc_recall_at_1_max
value: 50.55668152952304
- type: nauc_recall_at_1_std
value: 1.6572084853398048
- type: nauc_recall_at_20_diff1
value: 53.259157546141154
- type: nauc_recall_at_20_max
value: 54.03255118937038
- type: nauc_recall_at_20_std
value: 15.16161167427274
- type: nauc_recall_at_3_diff1
value: 60.72678574894387
- type: nauc_recall_at_3_max
value: 56.13989687586933
- type: nauc_recall_at_3_std
value: 2.2306901035770066
- type: nauc_recall_at_5_diff1
value: 57.12011275251864
- type: nauc_recall_at_5_max
value: 53.28665761862502
- type: nauc_recall_at_5_std
value: 4.3587200501122245
- type: ndcg_at_1
value: 30.0
- type: ndcg_at_10
value: 38.671
- type: ndcg_at_100
value: 42.173
- type: ndcg_at_1000
value: 44.016
- type: ndcg_at_20
value: 39.845000000000006
- type: ndcg_at_3
value: 36.863
- type: ndcg_at_5
value: 37.874
- type: precision_at_1
value: 30.0
- type: precision_at_10
value: 4.65
- type: precision_at_100
value: 0.64
- type: precision_at_1000
value: 0.08
- type: precision_at_20
value: 2.55
- type: precision_at_3
value: 13.833
- type: precision_at_5
value: 8.799999999999999
- type: recall_at_1
value: 30.0
- type: recall_at_10
value: 46.5
- type: recall_at_100
value: 64.0
- type: recall_at_1000
value: 79.5
- type: recall_at_20
value: 51.0
- type: recall_at_3
value: 41.5
- type: recall_at_5
value: 44.0
task:
type: Retrieval
- dataset:
config: rus
name: MTEB MultilingualSentimentClassification (rus)
revision: 2b9b4d10fc589af67794141fe8cbd3739de1eb33
split: test
type: mteb/multilingual-sentiment-classification
metrics:
- type: accuracy
value: 79.52710495963092
- type: ap
value: 84.5713457178972
- type: ap_weighted
value: 84.5713457178972
- type: f1
value: 77.88661181524105
- type: f1_weighted
value: 79.87563079922718
- type: main_score
value: 79.52710495963092
task:
type: Classification
- dataset:
config: arb_Arab-rus_Cyrl
name: MTEB NTREXBitextMining (arb_Arab-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 86.47971957936905
- type: f1
value: 82.79864240805654
- type: main_score
value: 82.79864240805654
- type: precision
value: 81.21485800128767
- type: recall
value: 86.47971957936905
task:
type: BitextMining
- dataset:
config: bel_Cyrl-rus_Cyrl
name: MTEB NTREXBitextMining (bel_Cyrl-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 94.84226339509264
- type: f1
value: 93.56399067465667
- type: main_score
value: 93.56399067465667
- type: precision
value: 93.01619095309631
- type: recall
value: 94.84226339509264
task:
type: BitextMining
- dataset:
config: ben_Beng-rus_Cyrl
name: MTEB NTREXBitextMining (ben_Beng-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 92.18828242363544
- type: f1
value: 90.42393889620612
- type: main_score
value: 90.42393889620612
- type: precision
value: 89.67904925153297
- type: recall
value: 92.18828242363544
task:
type: BitextMining
- dataset:
config: bos_Latn-rus_Cyrl
name: MTEB NTREXBitextMining (bos_Latn-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 94.69203805708563
- type: f1
value: 93.37172425304624
- type: main_score
value: 93.37172425304624
- type: precision
value: 92.79204521067315
- type: recall
value: 94.69203805708563
task:
type: BitextMining
- dataset:
config: bul_Cyrl-rus_Cyrl
name: MTEB NTREXBitextMining (bul_Cyrl-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 96.99549323985978
- type: f1
value: 96.13086296110833
- type: main_score
value: 96.13086296110833
- type: precision
value: 95.72441996327827
- type: recall
value: 96.99549323985978
task:
type: BitextMining
- dataset:
config: ces_Latn-rus_Cyrl
name: MTEB NTREXBitextMining (ces_Latn-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 95.94391587381071
- type: f1
value: 94.90680465142157
- type: main_score
value: 94.90680465142157
- type: precision
value: 94.44541812719079
- type: recall
value: 95.94391587381071
task:
type: BitextMining
- dataset:
config: deu_Latn-rus_Cyrl
name: MTEB NTREXBitextMining (deu_Latn-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 96.09414121181773
- type: f1
value: 94.94408279085295
- type: main_score
value: 94.94408279085295
- type: precision
value: 94.41245201135037
- type: recall
value: 96.09414121181773
task:
type: BitextMining
- dataset:
config: ell_Grek-rus_Cyrl
name: MTEB NTREXBitextMining (ell_Grek-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 96.19429143715573
- type: f1
value: 95.12101485561676
- type: main_score
value: 95.12101485561676
- type: precision
value: 94.60440660991488
- type: recall
value: 96.19429143715573
task:
type: BitextMining
- dataset:
config: eng_Latn-rus_Cyrl
name: MTEB NTREXBitextMining (eng_Latn-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 96.49474211316975
- type: f1
value: 95.46581777428045
- type: main_score
value: 95.46581777428045
- type: precision
value: 94.98414288098814
- type: recall
value: 96.49474211316975
task:
type: BitextMining
- dataset:
config: fas_Arab-rus_Cyrl
name: MTEB NTREXBitextMining (fas_Arab-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 94.44166249374061
- type: f1
value: 92.92383018972905
- type: main_score
value: 92.92383018972905
- type: precision
value: 92.21957936905358
- type: recall
value: 94.44166249374061
task:
type: BitextMining
- dataset:
config: fin_Latn-rus_Cyrl
name: MTEB NTREXBitextMining (fin_Latn-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 92.18828242363544
- type: f1
value: 90.2980661468393
- type: main_score
value: 90.2980661468393
- type: precision
value: 89.42580537472877
- type: recall
value: 92.18828242363544
task:
type: BitextMining
- dataset:
config: fra_Latn-rus_Cyrl
name: MTEB NTREXBitextMining (fra_Latn-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 95.84376564847271
- type: f1
value: 94.81054915706895
- type: main_score
value: 94.81054915706895
- type: precision
value: 94.31369276136427
- type: recall
value: 95.84376564847271
task:
type: BitextMining
- dataset:
config: heb_Hebr-rus_Cyrl
name: MTEB NTREXBitextMining (heb_Hebr-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 94.89233850776164
- type: f1
value: 93.42513770655985
- type: main_score
value: 93.42513770655985
- type: precision
value: 92.73493573693875
- type: recall
value: 94.89233850776164
task:
type: BitextMining
- dataset:
config: hin_Deva-rus_Cyrl
name: MTEB NTREXBitextMining (hin_Deva-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 93.23985978968453
- type: f1
value: 91.52816526376867
- type: main_score
value: 91.52816526376867
- type: precision
value: 90.76745946425466
- type: recall
value: 93.23985978968453
task:
type: BitextMining
- dataset:
config: hrv_Latn-rus_Cyrl
name: MTEB NTREXBitextMining (hrv_Latn-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 93.99098647971958
- type: f1
value: 92.36354531797697
- type: main_score
value: 92.36354531797697
- type: precision
value: 91.63228970439788
- type: recall
value: 93.99098647971958
task:
type: BitextMining
- dataset:
config: hun_Latn-rus_Cyrl
name: MTEB NTREXBitextMining (hun_Latn-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 93.64046069103655
- type: f1
value: 92.05224503421799
- type: main_score
value: 92.05224503421799
- type: precision
value: 91.33998616973079
- type: recall
value: 93.64046069103655
task:
type: BitextMining
- dataset:
config: ind_Latn-rus_Cyrl
name: MTEB NTREXBitextMining (ind_Latn-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 91.68753129694541
- type: f1
value: 89.26222667334335
- type: main_score
value: 89.26222667334335
- type: precision
value: 88.14638624603572
- type: recall
value: 91.68753129694541
task:
type: BitextMining
- dataset:
config: jpn_Jpan-rus_Cyrl
name: MTEB NTREXBitextMining (jpn_Jpan-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 91.28693039559339
- type: f1
value: 89.21161763348957
- type: main_score
value: 89.21161763348957
- type: precision
value: 88.31188340952988
- type: recall
value: 91.28693039559339
task:
type: BitextMining
- dataset:
config: kor_Hang-rus_Cyrl
name: MTEB NTREXBitextMining (kor_Hang-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 89.53430145217827
- type: f1
value: 86.88322165788365
- type: main_score
value: 86.88322165788365
- type: precision
value: 85.73950211030831
- type: recall
value: 89.53430145217827
task:
type: BitextMining
- dataset:
config: lit_Latn-rus_Cyrl
name: MTEB NTREXBitextMining (lit_Latn-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 90.28542814221332
- type: f1
value: 88.10249103814452
- type: main_score
value: 88.10249103814452
- type: precision
value: 87.17689323973752
- type: recall
value: 90.28542814221332
task:
type: BitextMining
- dataset:
config: mkd_Cyrl-rus_Cyrl
name: MTEB NTREXBitextMining (mkd_Cyrl-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 95.04256384576865
- type: f1
value: 93.65643703650713
- type: main_score
value: 93.65643703650713
- type: precision
value: 93.02036387915207
- type: recall
value: 95.04256384576865
task:
type: BitextMining
- dataset:
config: nld_Latn-rus_Cyrl
name: MTEB NTREXBitextMining (nld_Latn-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 95.39308963445168
- type: f1
value: 94.16207644800535
- type: main_score
value: 94.16207644800535
- type: precision
value: 93.582516632091
- type: recall
value: 95.39308963445168
task:
type: BitextMining
- dataset:
config: pol_Latn-rus_Cyrl
name: MTEB NTREXBitextMining (pol_Latn-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 95.7436154231347
- type: f1
value: 94.5067601402103
- type: main_score
value: 94.5067601402103
- type: precision
value: 93.91587381071608
- type: recall
value: 95.7436154231347
task:
type: BitextMining
- dataset:
config: por_Latn-rus_Cyrl
name: MTEB NTREXBitextMining (por_Latn-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 65.89884827240861
- type: f1
value: 64.61805459419219
- type: main_score
value: 64.61805459419219
- type: precision
value: 64.07119451106485
- type: recall
value: 65.89884827240861
task:
type: BitextMining
- dataset:
config: rus_Cyrl-arb_Arab
name: MTEB NTREXBitextMining (rus_Cyrl-arb_Arab)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 94.2413620430646
- type: f1
value: 92.67663399861698
- type: main_score
value: 92.67663399861698
- type: precision
value: 91.94625271240193
- type: recall
value: 94.2413620430646
task:
type: BitextMining
- dataset:
config: rus_Cyrl-bel_Cyrl
name: MTEB NTREXBitextMining (rus_Cyrl-bel_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 94.89233850776164
- type: f1
value: 93.40343849106993
- type: main_score
value: 93.40343849106993
- type: precision
value: 92.74077783341679
- type: recall
value: 94.89233850776164
task:
type: BitextMining
- dataset:
config: rus_Cyrl-ben_Beng
name: MTEB NTREXBitextMining (rus_Cyrl-ben_Beng)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 94.2914371557336
- type: f1
value: 92.62226673343348
- type: main_score
value: 92.62226673343348
- type: precision
value: 91.84610248706393
- type: recall
value: 94.2914371557336
task:
type: BitextMining
- dataset:
config: rus_Cyrl-bos_Latn
name: MTEB NTREXBitextMining (rus_Cyrl-bos_Latn)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 95.69354031046569
- type: f1
value: 94.50418051319403
- type: main_score
value: 94.50418051319403
- type: precision
value: 93.95843765648473
- type: recall
value: 95.69354031046569
task:
type: BitextMining
- dataset:
config: rus_Cyrl-bul_Cyrl
name: MTEB NTREXBitextMining (rus_Cyrl-bul_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 95.89384076114172
- type: f1
value: 94.66199298948423
- type: main_score
value: 94.66199298948423
- type: precision
value: 94.08028709731263
- type: recall
value: 95.89384076114172
task:
type: BitextMining
- dataset:
config: rus_Cyrl-ces_Latn
name: MTEB NTREXBitextMining (rus_Cyrl-ces_Latn)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 93.94091136705057
- type: f1
value: 92.3746731207923
- type: main_score
value: 92.3746731207923
- type: precision
value: 91.66207644800535
- type: recall
value: 93.94091136705057
task:
type: BitextMining
- dataset:
config: rus_Cyrl-deu_Latn
name: MTEB NTREXBitextMining (rus_Cyrl-deu_Latn)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 95.94391587381071
- type: f1
value: 94.76214321482223
- type: main_score
value: 94.76214321482223
- type: precision
value: 94.20380570856285
- type: recall
value: 95.94391587381071
task:
type: BitextMining
- dataset:
config: rus_Cyrl-ell_Grek
name: MTEB NTREXBitextMining (rus_Cyrl-ell_Grek)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 95.44316474712068
- type: f1
value: 94.14788849941579
- type: main_score
value: 94.14788849941579
- type: precision
value: 93.54197963612084
- type: recall
value: 95.44316474712068
task:
type: BitextMining
- dataset:
config: rus_Cyrl-eng_Latn
name: MTEB NTREXBitextMining (rus_Cyrl-eng_Latn)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 98.14722083124687
- type: f1
value: 97.57135703555333
- type: main_score
value: 97.57135703555333
- type: precision
value: 97.2959439158738
- type: recall
value: 98.14722083124687
task:
type: BitextMining
- dataset:
config: rus_Cyrl-fas_Arab
name: MTEB NTREXBitextMining (rus_Cyrl-fas_Arab)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 94.64196294441662
- type: f1
value: 93.24653647137372
- type: main_score
value: 93.24653647137372
- type: precision
value: 92.60724419963279
- type: recall
value: 94.64196294441662
task:
type: BitextMining
- dataset:
config: rus_Cyrl-fin_Latn
name: MTEB NTREXBitextMining (rus_Cyrl-fin_Latn)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 87.98197295943916
- type: f1
value: 85.23368385912201
- type: main_score
value: 85.23368385912201
- type: precision
value: 84.08159858835873
- type: recall
value: 87.98197295943916
task:
type: BitextMining
- dataset:
config: rus_Cyrl-fra_Latn
name: MTEB NTREXBitextMining (rus_Cyrl-fra_Latn)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 96.24436654982473
- type: f1
value: 95.07093974294774
- type: main_score
value: 95.07093974294774
- type: precision
value: 94.49591053246536
- type: recall
value: 96.24436654982473
task:
type: BitextMining
- dataset:
config: rus_Cyrl-heb_Hebr
name: MTEB NTREXBitextMining (rus_Cyrl-heb_Hebr)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 91.08662994491738
- type: f1
value: 88.5161074945752
- type: main_score
value: 88.5161074945752
- type: precision
value: 87.36187614755467
- type: recall
value: 91.08662994491738
task:
type: BitextMining
- dataset:
config: rus_Cyrl-hin_Deva
name: MTEB NTREXBitextMining (rus_Cyrl-hin_Deva)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 95.04256384576865
- type: f1
value: 93.66382907694876
- type: main_score
value: 93.66382907694876
- type: precision
value: 93.05291270238692
- type: recall
value: 95.04256384576865
task:
type: BitextMining
- dataset:
config: rus_Cyrl-hrv_Latn
name: MTEB NTREXBitextMining (rus_Cyrl-hrv_Latn)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 95.14271407110667
- type: f1
value: 93.7481221832749
- type: main_score
value: 93.7481221832749
- type: precision
value: 93.10930681736892
- type: recall
value: 95.14271407110667
task:
type: BitextMining
- dataset:
config: rus_Cyrl-hun_Latn
name: MTEB NTREXBitextMining (rus_Cyrl-hun_Latn)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 90.18527791687532
- type: f1
value: 87.61415933423946
- type: main_score
value: 87.61415933423946
- type: precision
value: 86.5166400394242
- type: recall
value: 90.18527791687532
task:
type: BitextMining
- dataset:
config: rus_Cyrl-ind_Latn
name: MTEB NTREXBitextMining (rus_Cyrl-ind_Latn)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 93.69053580370556
- type: f1
value: 91.83608746453012
- type: main_score
value: 91.83608746453012
- type: precision
value: 90.97145718577868
- type: recall
value: 93.69053580370556
task:
type: BitextMining
- dataset:
config: rus_Cyrl-jpn_Jpan
name: MTEB NTREXBitextMining (rus_Cyrl-jpn_Jpan)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 89.48422633950926
- type: f1
value: 86.91271033534429
- type: main_score
value: 86.91271033534429
- type: precision
value: 85.82671626487351
- type: recall
value: 89.48422633950926
task:
type: BitextMining
- dataset:
config: rus_Cyrl-kor_Hang
name: MTEB NTREXBitextMining (rus_Cyrl-kor_Hang)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 88.4827240861292
- type: f1
value: 85.35080398375342
- type: main_score
value: 85.35080398375342
- type: precision
value: 83.9588549490903
- type: recall
value: 88.4827240861292
task:
type: BitextMining
- dataset:
config: rus_Cyrl-lit_Latn
name: MTEB NTREXBitextMining (rus_Cyrl-lit_Latn)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 90.33550325488233
- type: f1
value: 87.68831819157307
- type: main_score
value: 87.68831819157307
- type: precision
value: 86.51524906407231
- type: recall
value: 90.33550325488233
task:
type: BitextMining
- dataset:
config: rus_Cyrl-mkd_Cyrl
name: MTEB NTREXBitextMining (rus_Cyrl-mkd_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 95.94391587381071
- type: f1
value: 94.90402270071775
- type: main_score
value: 94.90402270071775
- type: precision
value: 94.43915873810715
- type: recall
value: 95.94391587381071
task:
type: BitextMining
- dataset:
config: rus_Cyrl-nld_Latn
name: MTEB NTREXBitextMining (rus_Cyrl-nld_Latn)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 92.98948422633951
- type: f1
value: 91.04323151393756
- type: main_score
value: 91.04323151393756
- type: precision
value: 90.14688699716241
- type: recall
value: 92.98948422633951
task:
type: BitextMining
- dataset:
config: rus_Cyrl-pol_Latn
name: MTEB NTREXBitextMining (rus_Cyrl-pol_Latn)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 94.34151226840261
- type: f1
value: 92.8726422967785
- type: main_score
value: 92.8726422967785
- type: precision
value: 92.19829744616925
- type: recall
value: 94.34151226840261
task:
type: BitextMining
- dataset:
config: rus_Cyrl-por_Latn
name: MTEB NTREXBitextMining (rus_Cyrl-por_Latn)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 86.17926890335504
- type: f1
value: 82.7304882287356
- type: main_score
value: 82.7304882287356
- type: precision
value: 81.28162481817964
- type: recall
value: 86.17926890335504
task:
type: BitextMining
- dataset:
config: rus_Cyrl-slk_Latn
name: MTEB NTREXBitextMining (rus_Cyrl-slk_Latn)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 92.7391086629945
- type: f1
value: 90.75112669003506
- type: main_score
value: 90.75112669003506
- type: precision
value: 89.8564513436822
- type: recall
value: 92.7391086629945
task:
type: BitextMining
- dataset:
config: rus_Cyrl-slv_Latn
name: MTEB NTREXBitextMining (rus_Cyrl-slv_Latn)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 92.8893340010015
- type: f1
value: 91.05992321816058
- type: main_score
value: 91.05992321816058
- type: precision
value: 90.22589439715128
- type: recall
value: 92.8893340010015
task:
type: BitextMining
- dataset:
config: rus_Cyrl-spa_Latn
name: MTEB NTREXBitextMining (rus_Cyrl-spa_Latn)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 96.49474211316975
- type: f1
value: 95.4715406442998
- type: main_score
value: 95.4715406442998
- type: precision
value: 94.9799699549324
- type: recall
value: 96.49474211316975
task:
type: BitextMining
- dataset:
config: rus_Cyrl-srp_Cyrl
name: MTEB NTREXBitextMining (rus_Cyrl-srp_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 81.07160741111667
- type: f1
value: 76.55687285507015
- type: main_score
value: 76.55687285507015
- type: precision
value: 74.71886401030116
- type: recall
value: 81.07160741111667
task:
type: BitextMining
- dataset:
config: rus_Cyrl-srp_Latn
name: MTEB NTREXBitextMining (rus_Cyrl-srp_Latn)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 95.14271407110667
- type: f1
value: 93.73302377809138
- type: main_score
value: 93.73302377809138
- type: precision
value: 93.06960440660991
- type: recall
value: 95.14271407110667
task:
type: BitextMining
- dataset:
config: rus_Cyrl-swa_Latn
name: MTEB NTREXBitextMining (rus_Cyrl-swa_Latn)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 94.79218828242364
- type: f1
value: 93.25988983475212
- type: main_score
value: 93.25988983475212
- type: precision
value: 92.53463528626273
- type: recall
value: 94.79218828242364
task:
type: BitextMining
- dataset:
config: rus_Cyrl-swe_Latn
name: MTEB NTREXBitextMining (rus_Cyrl-swe_Latn)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 95.04256384576865
- type: f1
value: 93.58704723752295
- type: main_score
value: 93.58704723752295
- type: precision
value: 92.91437155733601
- type: recall
value: 95.04256384576865
task:
type: BitextMining
- dataset:
config: rus_Cyrl-tam_Taml
name: MTEB NTREXBitextMining (rus_Cyrl-tam_Taml)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 93.28993490235354
- type: f1
value: 91.63912535469872
- type: main_score
value: 91.63912535469872
- type: precision
value: 90.87738750983617
- type: recall
value: 93.28993490235354
task:
type: BitextMining
- dataset:
config: rus_Cyrl-tur_Latn
name: MTEB NTREXBitextMining (rus_Cyrl-tur_Latn)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 93.74061091637456
- type: f1
value: 91.96628275746953
- type: main_score
value: 91.96628275746953
- type: precision
value: 91.15923885828742
- type: recall
value: 93.74061091637456
task:
type: BitextMining
- dataset:
config: rus_Cyrl-ukr_Cyrl
name: MTEB NTREXBitextMining (rus_Cyrl-ukr_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 95.99399098647972
- type: f1
value: 94.89567684860624
- type: main_score
value: 94.89567684860624
- type: precision
value: 94.37072275079286
- type: recall
value: 95.99399098647972
task:
type: BitextMining
- dataset:
config: rus_Cyrl-vie_Latn
name: MTEB NTREXBitextMining (rus_Cyrl-vie_Latn)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 91.4371557336004
- type: f1
value: 88.98681355366382
- type: main_score
value: 88.98681355366382
- type: precision
value: 87.89183775663496
- type: recall
value: 91.4371557336004
task:
type: BitextMining
- dataset:
config: rus_Cyrl-zho_Hant
name: MTEB NTREXBitextMining (rus_Cyrl-zho_Hant)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 92.7891837756635
- type: f1
value: 90.79047142141783
- type: main_score
value: 90.79047142141783
- type: precision
value: 89.86980470706058
- type: recall
value: 92.7891837756635
task:
type: BitextMining
- dataset:
config: rus_Cyrl-zul_Latn
name: MTEB NTREXBitextMining (rus_Cyrl-zul_Latn)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 87.43114672008012
- type: f1
value: 84.04618833011422
- type: main_score
value: 84.04618833011422
- type: precision
value: 82.52259341393041
- type: recall
value: 87.43114672008012
task:
type: BitextMining
- dataset:
config: slk_Latn-rus_Cyrl
name: MTEB NTREXBitextMining (slk_Latn-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 95.34301452178268
- type: f1
value: 94.20392493502158
- type: main_score
value: 94.20392493502158
- type: precision
value: 93.67384409948257
- type: recall
value: 95.34301452178268
task:
type: BitextMining
- dataset:
config: slv_Latn-rus_Cyrl
name: MTEB NTREXBitextMining (slv_Latn-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 92.23835753630446
- type: f1
value: 90.5061759305625
- type: main_score
value: 90.5061759305625
- type: precision
value: 89.74231188051918
- type: recall
value: 92.23835753630446
task:
type: BitextMining
- dataset:
config: spa_Latn-rus_Cyrl
name: MTEB NTREXBitextMining (spa_Latn-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 96.54481722583876
- type: f1
value: 95.54665331330328
- type: main_score
value: 95.54665331330328
- type: precision
value: 95.06342847604739
- type: recall
value: 96.54481722583876
task:
type: BitextMining
- dataset:
config: srp_Cyrl-rus_Cyrl
name: MTEB NTREXBitextMining (srp_Cyrl-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 83.62543815723585
- type: f1
value: 80.77095672699816
- type: main_score
value: 80.77095672699816
- type: precision
value: 79.74674313056886
- type: recall
value: 83.62543815723585
task:
type: BitextMining
- dataset:
config: srp_Latn-rus_Cyrl
name: MTEB NTREXBitextMining (srp_Latn-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 94.44166249374061
- type: f1
value: 93.00733206591994
- type: main_score
value: 93.00733206591994
- type: precision
value: 92.37203026762366
- type: recall
value: 94.44166249374061
task:
type: BitextMining
- dataset:
config: swa_Latn-rus_Cyrl
name: MTEB NTREXBitextMining (swa_Latn-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 90.23535302954431
- type: f1
value: 87.89596482636041
- type: main_score
value: 87.89596482636041
- type: precision
value: 86.87060227370694
- type: recall
value: 90.23535302954431
task:
type: BitextMining
- dataset:
config: swe_Latn-rus_Cyrl
name: MTEB NTREXBitextMining (swe_Latn-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 95.44316474712068
- type: f1
value: 94.1896177599733
- type: main_score
value: 94.1896177599733
- type: precision
value: 93.61542313470206
- type: recall
value: 95.44316474712068
task:
type: BitextMining
- dataset:
config: tam_Taml-rus_Cyrl
name: MTEB NTREXBitextMining (tam_Taml-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 89.68452679018529
- type: f1
value: 87.37341160650037
- type: main_score
value: 87.37341160650037
- type: precision
value: 86.38389402285247
- type: recall
value: 89.68452679018529
task:
type: BitextMining
- dataset:
config: tur_Latn-rus_Cyrl
name: MTEB NTREXBitextMining (tur_Latn-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 93.89083625438157
- type: f1
value: 92.33892505424804
- type: main_score
value: 92.33892505424804
- type: precision
value: 91.63125640842216
- type: recall
value: 93.89083625438157
task:
type: BitextMining
- dataset:
config: ukr_Cyrl-rus_Cyrl
name: MTEB NTREXBitextMining (ukr_Cyrl-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 96.14421632448673
- type: f1
value: 95.11028447433054
- type: main_score
value: 95.11028447433054
- type: precision
value: 94.62944416624937
- type: recall
value: 96.14421632448673
task:
type: BitextMining
- dataset:
config: vie_Latn-rus_Cyrl
name: MTEB NTREXBitextMining (vie_Latn-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 93.79068602904357
- type: f1
value: 92.14989150392256
- type: main_score
value: 92.14989150392256
- type: precision
value: 91.39292271740945
- type: recall
value: 93.79068602904357
task:
type: BitextMining
- dataset:
config: zho_Hant-rus_Cyrl
name: MTEB NTREXBitextMining (zho_Hant-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 89.13370055082625
- type: f1
value: 86.51514618639217
- type: main_score
value: 86.51514618639217
- type: precision
value: 85.383920035898
- type: recall
value: 89.13370055082625
task:
type: BitextMining
- dataset:
config: zul_Latn-rus_Cyrl
name: MTEB NTREXBitextMining (zul_Latn-rus_Cyrl)
revision: ed9a4403ed4adbfaf4aab56d5b2709e9f6c3ba33
split: test
type: mteb/NTREX
metrics:
- type: accuracy
value: 81.17175763645467
- type: f1
value: 77.72331766047338
- type: main_score
value: 77.72331766047338
- type: precision
value: 76.24629555848075
- type: recall
value: 81.17175763645467
task:
type: BitextMining
- dataset:
config: ru
name: MTEB OpusparcusPC (ru)
revision: 9e9b1f8ef51616073f47f306f7f47dd91663f86a
split: test.full
type: GEM/opusparcus
metrics:
- type: cosine_accuracy
value: 73.09136420525657
- type: cosine_accuracy_threshold
value: 87.70400881767273
- type: cosine_ap
value: 86.51938550599533
- type: cosine_f1
value: 80.84358523725834
- type: cosine_f1_threshold
value: 86.90648078918457
- type: cosine_precision
value: 73.24840764331209
- type: cosine_recall
value: 90.19607843137256
- type: dot_accuracy
value: 73.09136420525657
- type: dot_accuracy_threshold
value: 87.7040147781372
- type: dot_ap
value: 86.51934769946833
- type: dot_f1
value: 80.84358523725834
- type: dot_f1_threshold
value: 86.90648078918457
- type: dot_precision
value: 73.24840764331209
- type: dot_recall
value: 90.19607843137256
- type: euclidean_accuracy
value: 73.09136420525657
- type: euclidean_accuracy_threshold
value: 49.590304493904114
- type: euclidean_ap
value: 86.51934769946833
- type: euclidean_f1
value: 80.84358523725834
- type: euclidean_f1_threshold
value: 51.173269748687744
- type: euclidean_precision
value: 73.24840764331209
- type: euclidean_recall
value: 90.19607843137256
- type: main_score
value: 86.51976811057995
- type: manhattan_accuracy
value: 73.40425531914893
- type: manhattan_accuracy_threshold
value: 757.8278541564941
- type: manhattan_ap
value: 86.51976811057995
- type: manhattan_f1
value: 80.92898615453328
- type: manhattan_f1_threshold
value: 778.3821105957031
- type: manhattan_precision
value: 74.32321575061526
- type: manhattan_recall
value: 88.8235294117647
- type: max_ap
value: 86.51976811057995
- type: max_f1
value: 80.92898615453328
- type: max_precision
value: 74.32321575061526
- type: max_recall
value: 90.19607843137256
- type: similarity_accuracy
value: 73.09136420525657
- type: similarity_accuracy_threshold
value: 87.70400881767273
- type: similarity_ap
value: 86.51938550599533
- type: similarity_f1
value: 80.84358523725834
- type: similarity_f1_threshold
value: 86.90648078918457
- type: similarity_precision
value: 73.24840764331209
- type: similarity_recall
value: 90.19607843137256
task:
type: PairClassification
- dataset:
config: russian
name: MTEB PublicHealthQA (russian)
revision: main
split: test
type: xhluca/publichealth-qa
metrics:
- type: main_score
value: 79.303
- type: map_at_1
value: 61.538000000000004
- type: map_at_10
value: 74.449
- type: map_at_100
value: 74.687
- type: map_at_1000
value: 74.687
- type: map_at_20
value: 74.589
- type: map_at_3
value: 73.333
- type: map_at_5
value: 74.256
- type: mrr_at_1
value: 61.53846153846154
- type: mrr_at_10
value: 74.44871794871794
- type: mrr_at_100
value: 74.68730304304074
- type: mrr_at_1000
value: 74.68730304304074
- type: mrr_at_20
value: 74.58857808857809
- type: mrr_at_3
value: 73.33333333333333
- type: mrr_at_5
value: 74.25641025641025
- type: nauc_map_at_1000_diff1
value: 61.375798048778506
- type: nauc_map_at_1000_max
value: 51.37093181241067
- type: nauc_map_at_1000_std
value: 41.735794471409015
- type: nauc_map_at_100_diff1
value: 61.375798048778506
- type: nauc_map_at_100_max
value: 51.37093181241067
- type: nauc_map_at_100_std
value: 41.735794471409015
- type: nauc_map_at_10_diff1
value: 61.12796039757213
- type: nauc_map_at_10_max
value: 51.843445267118014
- type: nauc_map_at_10_std
value: 42.243121474939365
- type: nauc_map_at_1_diff1
value: 66.39100974909151
- type: nauc_map_at_1_max
value: 44.77165601342703
- type: nauc_map_at_1_std
value: 32.38542979413408
- type: nauc_map_at_20_diff1
value: 61.16611123434347
- type: nauc_map_at_20_max
value: 51.52605092407306
- type: nauc_map_at_20_std
value: 41.94787773313971
- type: nauc_map_at_3_diff1
value: 61.40157474408937
- type: nauc_map_at_3_max
value: 51.47230077853947
- type: nauc_map_at_3_std
value: 42.63540269440141
- type: nauc_map_at_5_diff1
value: 61.07631147583098
- type: nauc_map_at_5_max
value: 52.02626939341523
- type: nauc_map_at_5_std
value: 42.511607332150334
- type: nauc_mrr_at_1000_diff1
value: 61.375798048778506
- type: nauc_mrr_at_1000_max
value: 51.37093181241067
- type: nauc_mrr_at_1000_std
value: 41.735794471409015
- type: nauc_mrr_at_100_diff1
value: 61.375798048778506
- type: nauc_mrr_at_100_max
value: 51.37093181241067
- type: nauc_mrr_at_100_std
value: 41.735794471409015
- type: nauc_mrr_at_10_diff1
value: 61.12796039757213
- type: nauc_mrr_at_10_max
value: 51.843445267118014
- type: nauc_mrr_at_10_std
value: 42.243121474939365
- type: nauc_mrr_at_1_diff1
value: 66.39100974909151
- type: nauc_mrr_at_1_max
value: 44.77165601342703
- type: nauc_mrr_at_1_std
value: 32.38542979413408
- type: nauc_mrr_at_20_diff1
value: 61.16611123434347
- type: nauc_mrr_at_20_max
value: 51.52605092407306
- type: nauc_mrr_at_20_std
value: 41.94787773313971
- type: nauc_mrr_at_3_diff1
value: 61.40157474408937
- type: nauc_mrr_at_3_max
value: 51.47230077853947
- type: nauc_mrr_at_3_std
value: 42.63540269440141
- type: nauc_mrr_at_5_diff1
value: 61.07631147583098
- type: nauc_mrr_at_5_max
value: 52.02626939341523
- type: nauc_mrr_at_5_std
value: 42.511607332150334
- type: nauc_ndcg_at_1000_diff1
value: 60.54821630436157
- type: nauc_ndcg_at_1000_max
value: 52.584328363863634
- type: nauc_ndcg_at_1000_std
value: 43.306961101645946
- type: nauc_ndcg_at_100_diff1
value: 60.54821630436157
- type: nauc_ndcg_at_100_max
value: 52.584328363863634
- type: nauc_ndcg_at_100_std
value: 43.306961101645946
- type: nauc_ndcg_at_10_diff1
value: 58.800340278109886
- type: nauc_ndcg_at_10_max
value: 55.31050771670664
- type: nauc_ndcg_at_10_std
value: 46.40931672942848
- type: nauc_ndcg_at_1_diff1
value: 66.39100974909151
- type: nauc_ndcg_at_1_max
value: 44.77165601342703
- type: nauc_ndcg_at_1_std
value: 32.38542979413408
- type: nauc_ndcg_at_20_diff1
value: 58.88690479697946
- type: nauc_ndcg_at_20_max
value: 54.19269661177923
- type: nauc_ndcg_at_20_std
value: 45.39305589413174
- type: nauc_ndcg_at_3_diff1
value: 59.61866351451574
- type: nauc_ndcg_at_3_max
value: 54.23992718744033
- type: nauc_ndcg_at_3_std
value: 46.997379274101
- type: nauc_ndcg_at_5_diff1
value: 58.70739588066225
- type: nauc_ndcg_at_5_max
value: 55.76766902539152
- type: nauc_ndcg_at_5_std
value: 47.10553115762958
- type: nauc_precision_at_1000_diff1
value: 100.0
- type: nauc_precision_at_1000_max
value: 100.0
- type: nauc_precision_at_1000_std
value: 100.0
- type: nauc_precision_at_100_diff1
value: .nan
- type: nauc_precision_at_100_max
value: .nan
- type: nauc_precision_at_100_std
value: .nan
- type: nauc_precision_at_10_diff1
value: 35.72622112397501
- type: nauc_precision_at_10_max
value: 89.84297108673948
- type: nauc_precision_at_10_std
value: 86.60269192422707
- type: nauc_precision_at_1_diff1
value: 66.39100974909151
- type: nauc_precision_at_1_max
value: 44.77165601342703
- type: nauc_precision_at_1_std
value: 32.38542979413408
- type: nauc_precision_at_20_diff1
value: 29.188449183726433
- type: nauc_precision_at_20_max
value: 86.45729478231968
- type: nauc_precision_at_20_std
value: 86.45729478231968
- type: nauc_precision_at_3_diff1
value: 50.294126629236224
- type: nauc_precision_at_3_max
value: 68.98223127174579
- type: nauc_precision_at_3_std
value: 70.31195520376356
- type: nauc_precision_at_5_diff1
value: 39.648884288124385
- type: nauc_precision_at_5_max
value: 86.3409770687935
- type: nauc_precision_at_5_std
value: 83.74875373878356
- type: nauc_recall_at_1000_diff1
value: .nan
- type: nauc_recall_at_1000_max
value: .nan
- type: nauc_recall_at_1000_std
value: .nan
- type: nauc_recall_at_100_diff1
value: .nan
- type: nauc_recall_at_100_max
value: .nan
- type: nauc_recall_at_100_std
value: .nan
- type: nauc_recall_at_10_diff1
value: 35.72622112397516
- type: nauc_recall_at_10_max
value: 89.84297108673968
- type: nauc_recall_at_10_std
value: 86.60269192422749
- type: nauc_recall_at_1_diff1
value: 66.39100974909151
- type: nauc_recall_at_1_max
value: 44.77165601342703
- type: nauc_recall_at_1_std
value: 32.38542979413408
- type: nauc_recall_at_20_diff1
value: 29.188449183726323
- type: nauc_recall_at_20_max
value: 86.45729478231985
- type: nauc_recall_at_20_std
value: 86.45729478231985
- type: nauc_recall_at_3_diff1
value: 50.29412662923603
- type: nauc_recall_at_3_max
value: 68.98223127174562
- type: nauc_recall_at_3_std
value: 70.31195520376346
- type: nauc_recall_at_5_diff1
value: 39.64888428812445
- type: nauc_recall_at_5_max
value: 86.34097706879359
- type: nauc_recall_at_5_std
value: 83.74875373878366
- type: ndcg_at_1
value: 61.538000000000004
- type: ndcg_at_10
value: 79.303
- type: ndcg_at_100
value: 80.557
- type: ndcg_at_1000
value: 80.557
- type: ndcg_at_20
value: 79.732
- type: ndcg_at_3
value: 77.033
- type: ndcg_at_5
value: 78.818
- type: precision_at_1
value: 61.538000000000004
- type: precision_at_10
value: 9.385
- type: precision_at_100
value: 1.0
- type: precision_at_1000
value: 0.1
- type: precision_at_20
value: 4.769
- type: precision_at_3
value: 29.231
- type: precision_at_5
value: 18.462
- type: recall_at_1
value: 61.538000000000004
- type: recall_at_10
value: 93.84599999999999
- type: recall_at_100
value: 100.0
- type: recall_at_1000
value: 100.0
- type: recall_at_20
value: 95.38499999999999
- type: recall_at_3
value: 87.69200000000001
- type: recall_at_5
value: 92.308
task:
type: Retrieval
- dataset:
config: default
name: MTEB RUParaPhraserSTS (default)
revision: 43265056790b8f7c59e0139acb4be0a8dad2c8f4
split: test
type: merionum/ru_paraphraser
metrics:
- type: cosine_pearson
value: 64.73554596215753
- type: cosine_spearman
value: 70.45849652271855
- type: euclidean_pearson
value: 68.08069844834267
- type: euclidean_spearman
value: 70.45854872959124
- type: main_score
value: 70.45849652271855
- type: manhattan_pearson
value: 67.88325986519624
- type: manhattan_spearman
value: 70.21131896834542
- type: pearson
value: 64.73554596215753
- type: spearman
value: 70.45849652271855
task:
type: STS
- dataset:
config: default
name: MTEB RiaNewsRetrieval (default)
revision: 82374b0bbacda6114f39ff9c5b925fa1512ca5d7
split: test
type: ai-forever/ria-news-retrieval
metrics:
- type: main_score
value: 70.00999999999999
- type: map_at_1
value: 55.97
- type: map_at_10
value: 65.59700000000001
- type: map_at_100
value: 66.057
- type: map_at_1000
value: 66.074
- type: map_at_20
value: 65.892
- type: map_at_3
value: 63.74999999999999
- type: map_at_5
value: 64.84299999999999
- type: mrr_at_1
value: 55.88999999999999
- type: mrr_at_10
value: 65.55873015872977
- type: mrr_at_100
value: 66.01891495129716
- type: mrr_at_1000
value: 66.03538391493299
- type: mrr_at_20
value: 65.85351193431555
- type: mrr_at_3
value: 63.7133333333329
- type: mrr_at_5
value: 64.80483333333268
- type: nauc_map_at_1000_diff1
value: 65.95332946436318
- type: nauc_map_at_1000_max
value: 28.21204156197811
- type: nauc_map_at_1000_std
value: -13.139245767083743
- type: nauc_map_at_100_diff1
value: 65.94763105024367
- type: nauc_map_at_100_max
value: 28.212832170078205
- type: nauc_map_at_100_std
value: -13.131425849370665
- type: nauc_map_at_10_diff1
value: 65.88455089448388
- type: nauc_map_at_10_max
value: 28.13555838776792
- type: nauc_map_at_10_std
value: -13.326989827081023
- type: nauc_map_at_1_diff1
value: 69.31275711813979
- type: nauc_map_at_1_max
value: 26.386708520283758
- type: nauc_map_at_1_std
value: -14.434616447245464
- type: nauc_map_at_20_diff1
value: 65.91227032605677
- type: nauc_map_at_20_max
value: 28.20538655600886
- type: nauc_map_at_20_std
value: -13.191148834410274
- type: nauc_map_at_3_diff1
value: 66.0051677952641
- type: nauc_map_at_3_max
value: 28.25443420019022
- type: nauc_map_at_3_std
value: -13.893284109029558
- type: nauc_map_at_5_diff1
value: 65.89784348297898
- type: nauc_map_at_5_max
value: 28.26449765184183
- type: nauc_map_at_5_std
value: -13.506692912805008
- type: nauc_mrr_at_1000_diff1
value: 66.06599513750889
- type: nauc_mrr_at_1000_max
value: 28.191556650722287
- type: nauc_mrr_at_1000_std
value: -13.098487982930276
- type: nauc_mrr_at_100_diff1
value: 66.0602307977725
- type: nauc_mrr_at_100_max
value: 28.19235936624514
- type: nauc_mrr_at_100_std
value: -13.09069677716269
- type: nauc_mrr_at_10_diff1
value: 65.99546819079403
- type: nauc_mrr_at_10_max
value: 28.11556170120022
- type: nauc_mrr_at_10_std
value: -13.286711073897553
- type: nauc_mrr_at_1_diff1
value: 69.49541040517995
- type: nauc_mrr_at_1_max
value: 26.354622707276153
- type: nauc_mrr_at_1_std
value: -14.358839778104695
- type: nauc_mrr_at_20_diff1
value: 66.02427154257936
- type: nauc_mrr_at_20_max
value: 28.18509383563462
- type: nauc_mrr_at_20_std
value: -13.150543398429
- type: nauc_mrr_at_3_diff1
value: 66.11258119082618
- type: nauc_mrr_at_3_max
value: 28.239510722224004
- type: nauc_mrr_at_3_std
value: -13.857249251136269
- type: nauc_mrr_at_5_diff1
value: 66.00633786765626
- type: nauc_mrr_at_5_max
value: 28.244875152193032
- type: nauc_mrr_at_5_std
value: -13.467206028704434
- type: nauc_ndcg_at_1000_diff1
value: 65.02876183314446
- type: nauc_ndcg_at_1000_max
value: 29.109368390197194
- type: nauc_ndcg_at_1000_std
value: -11.56514359821697
- type: nauc_ndcg_at_100_diff1
value: 64.85837726893713
- type: nauc_ndcg_at_100_max
value: 29.19990133137256
- type: nauc_ndcg_at_100_std
value: -11.17450348161257
- type: nauc_ndcg_at_10_diff1
value: 64.53842705024796
- type: nauc_ndcg_at_10_max
value: 28.748734006088526
- type: nauc_ndcg_at_10_std
value: -12.331395505957063
- type: nauc_ndcg_at_1_diff1
value: 69.31275711813979
- type: nauc_ndcg_at_1_max
value: 26.386708520283758
- type: nauc_ndcg_at_1_std
value: -14.434616447245464
- type: nauc_ndcg_at_20_diff1
value: 64.59017606740504
- type: nauc_ndcg_at_20_max
value: 29.047332048898017
- type: nauc_ndcg_at_20_std
value: -11.746548770195954
- type: nauc_ndcg_at_3_diff1
value: 64.87900935713822
- type: nauc_ndcg_at_3_max
value: 28.953157521204403
- type: nauc_ndcg_at_3_std
value: -13.639947228880942
- type: nauc_ndcg_at_5_diff1
value: 64.61466953479034
- type: nauc_ndcg_at_5_max
value: 29.01899321868392
- type: nauc_ndcg_at_5_std
value: -12.85356404799802
- type: nauc_precision_at_1000_diff1
value: 48.85481417002382
- type: nauc_precision_at_1000_max
value: 57.129837326696375
- type: nauc_precision_at_1000_std
value: 37.889524999906435
- type: nauc_precision_at_100_diff1
value: 53.374672326788264
- type: nauc_precision_at_100_max
value: 43.819333062207974
- type: nauc_precision_at_100_std
value: 21.387064885769362
- type: nauc_precision_at_10_diff1
value: 57.66571169774445
- type: nauc_precision_at_10_max
value: 31.779694837242033
- type: nauc_precision_at_10_std
value: -6.6248399147180255
- type: nauc_precision_at_1_diff1
value: 69.31275711813979
- type: nauc_precision_at_1_max
value: 26.386708520283758
- type: nauc_precision_at_1_std
value: -14.434616447245464
- type: nauc_precision_at_20_diff1
value: 55.93570036001682
- type: nauc_precision_at_20_max
value: 34.98640173388743
- type: nauc_precision_at_20_std
value: -0.36518465159326174
- type: nauc_precision_at_3_diff1
value: 60.94100093991508
- type: nauc_precision_at_3_max
value: 31.422239034357673
- type: nauc_precision_at_3_std
value: -12.72576556537896
- type: nauc_precision_at_5_diff1
value: 59.450505195434054
- type: nauc_precision_at_5_max
value: 32.07638712418377
- type: nauc_precision_at_5_std
value: -10.024459103498598
- type: nauc_recall_at_1000_diff1
value: 48.854814170024184
- type: nauc_recall_at_1000_max
value: 57.129837326697164
- type: nauc_recall_at_1000_std
value: 37.88952499990672
- type: nauc_recall_at_100_diff1
value: 53.37467232678822
- type: nauc_recall_at_100_max
value: 43.8193330622079
- type: nauc_recall_at_100_std
value: 21.387064885769398
- type: nauc_recall_at_10_diff1
value: 57.66571169774447
- type: nauc_recall_at_10_max
value: 31.779694837242133
- type: nauc_recall_at_10_std
value: -6.62483991471789
- type: nauc_recall_at_1_diff1
value: 69.31275711813979
- type: nauc_recall_at_1_max
value: 26.386708520283758
- type: nauc_recall_at_1_std
value: -14.434616447245464
- type: nauc_recall_at_20_diff1
value: 55.93570036001682
- type: nauc_recall_at_20_max
value: 34.986401733887554
- type: nauc_recall_at_20_std
value: -0.3651846515931506
- type: nauc_recall_at_3_diff1
value: 60.94100093991499
- type: nauc_recall_at_3_max
value: 31.422239034357606
- type: nauc_recall_at_3_std
value: -12.725765565378966
- type: nauc_recall_at_5_diff1
value: 59.450505195434125
- type: nauc_recall_at_5_max
value: 32.07638712418387
- type: nauc_recall_at_5_std
value: -10.024459103498472
- type: ndcg_at_1
value: 55.97
- type: ndcg_at_10
value: 70.00999999999999
- type: ndcg_at_100
value: 72.20100000000001
- type: ndcg_at_1000
value: 72.65599999999999
- type: ndcg_at_20
value: 71.068
- type: ndcg_at_3
value: 66.228
- type: ndcg_at_5
value: 68.191
- type: precision_at_1
value: 55.97
- type: precision_at_10
value: 8.373999999999999
- type: precision_at_100
value: 0.9390000000000001
- type: precision_at_1000
value: 0.097
- type: precision_at_20
value: 4.3950000000000005
- type: precision_at_3
value: 24.46
- type: precision_at_5
value: 15.626000000000001
- type: recall_at_1
value: 55.97
- type: recall_at_10
value: 83.74000000000001
- type: recall_at_100
value: 93.87
- type: recall_at_1000
value: 97.49
- type: recall_at_20
value: 87.89
- type: recall_at_3
value: 73.38
- type: recall_at_5
value: 78.13
task:
type: Retrieval
- dataset:
config: default
name: MTEB RuBQReranking (default)
revision: 2e96b8f098fa4b0950fc58eacadeb31c0d0c7fa2
split: test
type: ai-forever/rubq-reranking
metrics:
- type: main_score
value: 71.44929565043827
- type: map
value: 71.44929565043827
- type: mrr
value: 77.78391820945014
- type: nAUC_map_diff1
value: 38.140840668080244
- type: nAUC_map_max
value: 27.54328688105381
- type: nAUC_map_std
value: 16.81572082284672
- type: nAUC_mrr_diff1
value: 44.51350415961509
- type: nAUC_mrr_max
value: 36.491182016669754
- type: nAUC_mrr_std
value: 22.47139593052269
task:
type: Reranking
- dataset:
config: default
name: MTEB RuBQRetrieval (default)
revision: e19b6ffa60b3bc248e0b41f4cc37c26a55c2a67b
split: test
type: ai-forever/rubq-retrieval
metrics:
- type: main_score
value: 68.529
- type: map_at_1
value: 42.529
- type: map_at_10
value: 60.864
- type: map_at_100
value: 61.868
- type: map_at_1000
value: 61.907000000000004
- type: map_at_20
value: 61.596
- type: map_at_3
value: 55.701
- type: map_at_5
value: 58.78
- type: mrr_at_1
value: 60.57919621749409
- type: mrr_at_10
value: 70.55614188149649
- type: mrr_at_100
value: 70.88383816664494
- type: mrr_at_1000
value: 70.89719252668833
- type: mrr_at_20
value: 70.79839750105347
- type: mrr_at_3
value: 68.4594168636722
- type: mrr_at_5
value: 69.67100078802214
- type: nauc_map_at_1000_diff1
value: 40.67438785660885
- type: nauc_map_at_1000_max
value: 32.79981738507424
- type: nauc_map_at_1000_std
value: -6.873402600044831
- type: nauc_map_at_100_diff1
value: 40.65643664443284
- type: nauc_map_at_100_max
value: 32.81594799919249
- type: nauc_map_at_100_std
value: -6.8473246794498195
- type: nauc_map_at_10_diff1
value: 40.39048268484908
- type: nauc_map_at_10_max
value: 32.403242161479525
- type: nauc_map_at_10_std
value: -7.344413799841244
- type: nauc_map_at_1_diff1
value: 44.36306892906905
- type: nauc_map_at_1_max
value: 25.61348630699028
- type: nauc_map_at_1_std
value: -8.713074613333902
- type: nauc_map_at_20_diff1
value: 40.530326570124615
- type: nauc_map_at_20_max
value: 32.74028319323205
- type: nauc_map_at_20_std
value: -7.008180779820569
- type: nauc_map_at_3_diff1
value: 40.764924859364044
- type: nauc_map_at_3_max
value: 29.809671682025336
- type: nauc_map_at_3_std
value: -9.205620202725564
- type: nauc_map_at_5_diff1
value: 40.88599496021476
- type: nauc_map_at_5_max
value: 32.1701894666848
- type: nauc_map_at_5_std
value: -7.801251849010623
- type: nauc_mrr_at_1000_diff1
value: 48.64181373540728
- type: nauc_mrr_at_1000_max
value: 40.136947990653546
- type: nauc_mrr_at_1000_std
value: -7.250260497468805
- type: nauc_mrr_at_100_diff1
value: 48.63349902496212
- type: nauc_mrr_at_100_max
value: 40.14510559704008
- type: nauc_mrr_at_100_std
value: -7.228702374801103
- type: nauc_mrr_at_10_diff1
value: 48.58580560194813
- type: nauc_mrr_at_10_max
value: 40.15075599433366
- type: nauc_mrr_at_10_std
value: -7.267928771548688
- type: nauc_mrr_at_1_diff1
value: 51.47535097164919
- type: nauc_mrr_at_1_max
value: 38.23579750430856
- type: nauc_mrr_at_1_std
value: -9.187785187137633
- type: nauc_mrr_at_20_diff1
value: 48.58688378336222
- type: nauc_mrr_at_20_max
value: 40.13408744088299
- type: nauc_mrr_at_20_std
value: -7.283132775160146
- type: nauc_mrr_at_3_diff1
value: 48.66833005454742
- type: nauc_mrr_at_3_max
value: 40.07987333638038
- type: nauc_mrr_at_3_std
value: -7.738819947521418
- type: nauc_mrr_at_5_diff1
value: 48.76536305941537
- type: nauc_mrr_at_5_max
value: 40.381929739522185
- type: nauc_mrr_at_5_std
value: -7.592858318378928
- type: nauc_ndcg_at_1000_diff1
value: 41.67304442004693
- type: nauc_ndcg_at_1000_max
value: 35.84126926253235
- type: nauc_ndcg_at_1000_std
value: -4.78971011604655
- type: nauc_ndcg_at_100_diff1
value: 41.16918850185783
- type: nauc_ndcg_at_100_max
value: 36.082461962326505
- type: nauc_ndcg_at_100_std
value: -4.092442251697269
- type: nauc_ndcg_at_10_diff1
value: 40.300065598615205
- type: nauc_ndcg_at_10_max
value: 34.87866296788365
- type: nauc_ndcg_at_10_std
value: -5.866529277842453
- type: nauc_ndcg_at_1_diff1
value: 51.74612915209495
- type: nauc_ndcg_at_1_max
value: 37.71907067970078
- type: nauc_ndcg_at_1_std
value: -9.064124266098696
- type: nauc_ndcg_at_20_diff1
value: 40.493949850214584
- type: nauc_ndcg_at_20_max
value: 35.69331503650286
- type: nauc_ndcg_at_20_std
value: -4.995310342975443
- type: nauc_ndcg_at_3_diff1
value: 41.269443212112364
- type: nauc_ndcg_at_3_max
value: 32.572844460953334
- type: nauc_ndcg_at_3_std
value: -9.063015396458791
- type: nauc_ndcg_at_5_diff1
value: 41.37039652522888
- type: nauc_ndcg_at_5_max
value: 34.67416011393571
- type: nauc_ndcg_at_5_std
value: -7.106845569862319
- type: nauc_precision_at_1000_diff1
value: -9.571769961090155
- type: nauc_precision_at_1000_max
value: 5.574782583417188
- type: nauc_precision_at_1000_std
value: 7.28333847923847
- type: nauc_precision_at_100_diff1
value: -7.7405012003383735
- type: nauc_precision_at_100_max
value: 9.67745355070353
- type: nauc_precision_at_100_std
value: 9.327890294080992
- type: nauc_precision_at_10_diff1
value: -1.006879647532931
- type: nauc_precision_at_10_max
value: 15.899825481231064
- type: nauc_precision_at_10_std
value: 4.2284084852153105
- type: nauc_precision_at_1_diff1
value: 51.74612915209495
- type: nauc_precision_at_1_max
value: 37.71907067970078
- type: nauc_precision_at_1_std
value: -9.064124266098696
- type: nauc_precision_at_20_diff1
value: -4.982301544401409
- type: nauc_precision_at_20_max
value: 13.241674471380568
- type: nauc_precision_at_20_std
value: 7.052280133821539
- type: nauc_precision_at_3_diff1
value: 15.442614376387374
- type: nauc_precision_at_3_max
value: 25.12695418083
- type: nauc_precision_at_3_std
value: -3.1150066697920638
- type: nauc_precision_at_5_diff1
value: 8.381026072692444
- type: nauc_precision_at_5_max
value: 22.839056540604822
- type: nauc_precision_at_5_std
value: 1.5126905486524331
- type: nauc_recall_at_1000_diff1
value: -0.8869709920433502
- type: nauc_recall_at_1000_max
value: 45.092324433377264
- type: nauc_recall_at_1000_std
value: 62.21264093315108
- type: nauc_recall_at_100_diff1
value: 16.036715011075714
- type: nauc_recall_at_100_max
value: 39.79963411771158
- type: nauc_recall_at_100_std
value: 28.41850069503361
- type: nauc_recall_at_10_diff1
value: 25.189622794479998
- type: nauc_recall_at_10_max
value: 30.82355277039427
- type: nauc_recall_at_10_std
value: 0.0964544736531047
- type: nauc_recall_at_1_diff1
value: 44.36306892906905
- type: nauc_recall_at_1_max
value: 25.61348630699028
- type: nauc_recall_at_1_std
value: -8.713074613333902
- type: nauc_recall_at_20_diff1
value: 20.43424504746087
- type: nauc_recall_at_20_max
value: 33.96010554649377
- type: nauc_recall_at_20_std
value: 6.900984030301936
- type: nauc_recall_at_3_diff1
value: 33.86531858793492
- type: nauc_recall_at_3_max
value: 27.725692256711188
- type: nauc_recall_at_3_std
value: -8.533124289305709
- type: nauc_recall_at_5_diff1
value: 32.006964557701686
- type: nauc_recall_at_5_max
value: 31.493370659289806
- type: nauc_recall_at_5_std
value: -4.8639793547793255
- type: ndcg_at_1
value: 60.461
- type: ndcg_at_10
value: 68.529
- type: ndcg_at_100
value: 71.664
- type: ndcg_at_1000
value: 72.396
- type: ndcg_at_20
value: 70.344
- type: ndcg_at_3
value: 61.550000000000004
- type: ndcg_at_5
value: 64.948
- type: precision_at_1
value: 60.461
- type: precision_at_10
value: 13.28
- type: precision_at_100
value: 1.555
- type: precision_at_1000
value: 0.164
- type: precision_at_20
value: 7.216
- type: precision_at_3
value: 33.077
- type: precision_at_5
value: 23.014000000000003
- type: recall_at_1
value: 42.529
- type: recall_at_10
value: 81.169
- type: recall_at_100
value: 93.154
- type: recall_at_1000
value: 98.18299999999999
- type: recall_at_20
value: 87.132
- type: recall_at_3
value: 63.905
- type: recall_at_5
value: 71.967
task:
type: Retrieval
- dataset:
config: default
name: MTEB RuReviewsClassification (default)
revision: f6d2c31f4dc6b88f468552750bfec05b4b41b05a
split: test
type: ai-forever/ru-reviews-classification
metrics:
- type: accuracy
value: 61.17675781250001
- type: f1
value: 60.354535346041374
- type: f1_weighted
value: 60.35437313166116
- type: main_score
value: 61.17675781250001
task:
type: Classification
- dataset:
config: default
name: MTEB RuSTSBenchmarkSTS (default)
revision: 7cf24f325c6da6195df55bef3d86b5e0616f3018
split: test
type: ai-forever/ru-stsbenchmark-sts
metrics:
- type: cosine_pearson
value: 78.1301041727274
- type: cosine_spearman
value: 78.08238025421747
- type: euclidean_pearson
value: 77.35224254583635
- type: euclidean_spearman
value: 78.08235336582496
- type: main_score
value: 78.08238025421747
- type: manhattan_pearson
value: 77.24138550052075
- type: manhattan_spearman
value: 77.98199107904142
- type: pearson
value: 78.1301041727274
- type: spearman
value: 78.08238025421747
task:
type: STS
- dataset:
config: default
name: MTEB RuSciBenchGRNTIClassification (default)
revision: 673a610d6d3dd91a547a0d57ae1b56f37ebbf6a1
split: test
type: ai-forever/ru-scibench-grnti-classification
metrics:
- type: accuracy
value: 54.990234375
- type: f1
value: 53.537019057131374
- type: f1_weighted
value: 53.552745354520766
- type: main_score
value: 54.990234375
task:
type: Classification
- dataset:
config: default
name: MTEB RuSciBenchGRNTIClusteringP2P (default)
revision: 673a610d6d3dd91a547a0d57ae1b56f37ebbf6a1
split: test
type: ai-forever/ru-scibench-grnti-classification
metrics:
- type: main_score
value: 50.775228895355106
- type: v_measure
value: 50.775228895355106
- type: v_measure_std
value: 0.9533571150165796
task:
type: Clustering
- dataset:
config: default
name: MTEB RuSciBenchOECDClassification (default)
revision: 26c88e99dcaba32bb45d0e1bfc21902337f6d471
split: test
type: ai-forever/ru-scibench-oecd-classification
metrics:
- type: accuracy
value: 41.71875
- type: f1
value: 39.289100975858304
- type: f1_weighted
value: 39.29257829217775
- type: main_score
value: 41.71875
task:
type: Classification
- dataset:
config: default
name: MTEB RuSciBenchOECDClusteringP2P (default)
revision: 26c88e99dcaba32bb45d0e1bfc21902337f6d471
split: test
type: ai-forever/ru-scibench-oecd-classification
metrics:
- type: main_score
value: 45.10904808834516
- type: v_measure
value: 45.10904808834516
- type: v_measure_std
value: 1.0572643410157534
task:
type: Clustering
- dataset:
config: rus_Cyrl
name: MTEB SIB200Classification (rus_Cyrl)
revision: a74d7350ea12af010cfb1c21e34f1f81fd2e615b
split: test
type: mteb/sib200
metrics:
- type: accuracy
value: 66.36363636363637
- type: f1
value: 64.6940336621617
- type: f1_weighted
value: 66.43317771876966
- type: main_score
value: 66.36363636363637
task:
type: Classification
- dataset:
config: rus_Cyrl
name: MTEB SIB200ClusteringS2S (rus_Cyrl)
revision: a74d7350ea12af010cfb1c21e34f1f81fd2e615b
split: test
type: mteb/sib200
metrics:
- type: main_score
value: 33.99178497314711
- type: v_measure
value: 33.99178497314711
- type: v_measure_std
value: 4.036337464043786
task:
type: Clustering
- dataset:
config: ru
name: MTEB STS22.v2 (ru)
revision: d31f33a128469b20e357535c39b82fb3c3f6f2bd
split: test
type: mteb/sts22-crosslingual-sts
metrics:
- type: cosine_pearson
value: 50.724322379215934
- type: cosine_spearman
value: 59.90449732164651
- type: euclidean_pearson
value: 50.227545226784024
- type: euclidean_spearman
value: 59.898906527601085
- type: main_score
value: 59.90449732164651
- type: manhattan_pearson
value: 50.21762139819405
- type: manhattan_spearman
value: 59.761039813759
- type: pearson
value: 50.724322379215934
- type: spearman
value: 59.90449732164651
task:
type: STS
- dataset:
config: ru
name: MTEB STSBenchmarkMultilingualSTS (ru)
revision: 29afa2569dcedaaa2fe6a3dcfebab33d28b82e8c
split: dev
type: mteb/stsb_multi_mt
metrics:
- type: cosine_pearson
value: 78.43928769569945
- type: cosine_spearman
value: 78.23961768018884
- type: euclidean_pearson
value: 77.4718694027985
- type: euclidean_spearman
value: 78.23887044760475
- type: main_score
value: 78.23961768018884
- type: manhattan_pearson
value: 77.34517128089547
- type: manhattan_spearman
value: 78.1146477340426
- type: pearson
value: 78.43928769569945
- type: spearman
value: 78.23961768018884
task:
type: STS
- dataset:
config: default
name: MTEB SensitiveTopicsClassification (default)
revision: 416b34a802308eac30e4192afc0ff99bb8dcc7f2
split: test
type: ai-forever/sensitive-topics-classification
metrics:
- type: accuracy
value: 22.8125
- type: f1
value: 17.31969589593409
- type: lrap
value: 33.82412380642287
- type: main_score
value: 22.8125
task:
type: MultilabelClassification
- dataset:
config: default
name: MTEB TERRa (default)
revision: 7b58f24536063837d644aab9a023c62199b2a612
split: dev
type: ai-forever/terra-pairclassification
metrics:
- type: cosine_accuracy
value: 57.32899022801303
- type: cosine_accuracy_threshold
value: 85.32201051712036
- type: cosine_ap
value: 55.14264553720072
- type: cosine_f1
value: 66.83544303797468
- type: cosine_f1_threshold
value: 85.32201051712036
- type: cosine_precision
value: 54.54545454545454
- type: cosine_recall
value: 86.27450980392157
- type: dot_accuracy
value: 57.32899022801303
- type: dot_accuracy_threshold
value: 85.32201051712036
- type: dot_ap
value: 55.14264553720072
- type: dot_f1
value: 66.83544303797468
- type: dot_f1_threshold
value: 85.32201051712036
- type: dot_precision
value: 54.54545454545454
- type: dot_recall
value: 86.27450980392157
- type: euclidean_accuracy
value: 57.32899022801303
- type: euclidean_accuracy_threshold
value: 54.18117046356201
- type: euclidean_ap
value: 55.14264553720072
- type: euclidean_f1
value: 66.83544303797468
- type: euclidean_f1_threshold
value: 54.18117046356201
- type: euclidean_precision
value: 54.54545454545454
- type: euclidean_recall
value: 86.27450980392157
- type: main_score
value: 55.14264553720072
- type: manhattan_accuracy
value: 57.32899022801303
- type: manhattan_accuracy_threshold
value: 828.8480758666992
- type: manhattan_ap
value: 55.077974053622555
- type: manhattan_f1
value: 66.82352941176471
- type: manhattan_f1_threshold
value: 885.6784820556641
- type: manhattan_precision
value: 52.20588235294118
- type: manhattan_recall
value: 92.81045751633987
- type: max_ap
value: 55.14264553720072
- type: max_f1
value: 66.83544303797468
- type: max_precision
value: 54.54545454545454
- type: max_recall
value: 92.81045751633987
- type: similarity_accuracy
value: 57.32899022801303
- type: similarity_accuracy_threshold
value: 85.32201051712036
- type: similarity_ap
value: 55.14264553720072
- type: similarity_f1
value: 66.83544303797468
- type: similarity_f1_threshold
value: 85.32201051712036
- type: similarity_precision
value: 54.54545454545454
- type: similarity_recall
value: 86.27450980392157
task:
type: PairClassification
- dataset:
config: ru
name: MTEB XNLI (ru)
revision: 09698e0180d87dc247ca447d3a1248b931ac0cdb
split: test
type: mteb/xnli
metrics:
- type: cosine_accuracy
value: 67.6923076923077
- type: cosine_accuracy_threshold
value: 87.6681923866272
- type: cosine_ap
value: 73.18693800863593
- type: cosine_f1
value: 70.40641099026904
- type: cosine_f1_threshold
value: 85.09706258773804
- type: cosine_precision
value: 57.74647887323944
- type: cosine_recall
value: 90.17595307917888
- type: dot_accuracy
value: 67.6923076923077
- type: dot_accuracy_threshold
value: 87.66818642616272
- type: dot_ap
value: 73.18693800863593
- type: dot_f1
value: 70.40641099026904
- type: dot_f1_threshold
value: 85.09706258773804
- type: dot_precision
value: 57.74647887323944
- type: dot_recall
value: 90.17595307917888
- type: euclidean_accuracy
value: 67.6923076923077
- type: euclidean_accuracy_threshold
value: 49.662476778030396
- type: euclidean_ap
value: 73.18693800863593
- type: euclidean_f1
value: 70.40641099026904
- type: euclidean_f1_threshold
value: 54.59475517272949
- type: euclidean_precision
value: 57.74647887323944
- type: euclidean_recall
value: 90.17595307917888
- type: main_score
value: 73.18693800863593
- type: manhattan_accuracy
value: 67.54578754578755
- type: manhattan_accuracy_threshold
value: 777.1001815795898
- type: manhattan_ap
value: 72.98861474758783
- type: manhattan_f1
value: 70.6842435655995
- type: manhattan_f1_threshold
value: 810.3782653808594
- type: manhattan_precision
value: 61.80021953896817
- type: manhattan_recall
value: 82.55131964809385
- type: max_ap
value: 73.18693800863593
- type: max_f1
value: 70.6842435655995
- type: max_precision
value: 61.80021953896817
- type: max_recall
value: 90.17595307917888
- type: similarity_accuracy
value: 67.6923076923077
- type: similarity_accuracy_threshold
value: 87.6681923866272
- type: similarity_ap
value: 73.18693800863593
- type: similarity_f1
value: 70.40641099026904
- type: similarity_f1_threshold
value: 85.09706258773804
- type: similarity_precision
value: 57.74647887323944
- type: similarity_recall
value: 90.17595307917888
task:
type: PairClassification
- dataset:
config: russian
name: MTEB XNLIV2 (russian)
revision: 5b7d477a8c62cdd18e2fed7e015497c20b4371ad
split: test
type: mteb/xnli2.0-multi-pair
metrics:
- type: cosine_accuracy
value: 68.35164835164835
- type: cosine_accuracy_threshold
value: 88.48621845245361
- type: cosine_ap
value: 73.10205506215699
- type: cosine_f1
value: 71.28712871287128
- type: cosine_f1_threshold
value: 87.00399398803711
- type: cosine_precision
value: 61.67023554603854
- type: cosine_recall
value: 84.4574780058651
- type: dot_accuracy
value: 68.35164835164835
- type: dot_accuracy_threshold
value: 88.48622441291809
- type: dot_ap
value: 73.10191110714706
- type: dot_f1
value: 71.28712871287128
- type: dot_f1_threshold
value: 87.00399398803711
- type: dot_precision
value: 61.67023554603854
- type: dot_recall
value: 84.4574780058651
- type: euclidean_accuracy
value: 68.35164835164835
- type: euclidean_accuracy_threshold
value: 47.98704385757446
- type: euclidean_ap
value: 73.10205506215699
- type: euclidean_f1
value: 71.28712871287128
- type: euclidean_f1_threshold
value: 50.982362031936646
- type: euclidean_precision
value: 61.67023554603854
- type: euclidean_recall
value: 84.4574780058651
- type: main_score
value: 73.10205506215699
- type: manhattan_accuracy
value: 67.91208791208791
- type: manhattan_accuracy_threshold
value: 746.1360931396484
- type: manhattan_ap
value: 72.8954736175069
- type: manhattan_f1
value: 71.1297071129707
- type: manhattan_f1_threshold
value: 808.0789566040039
- type: manhattan_precision
value: 60.04036326942482
- type: manhattan_recall
value: 87.2434017595308
- type: max_ap
value: 73.10205506215699
- type: max_f1
value: 71.28712871287128
- type: max_precision
value: 61.67023554603854
- type: max_recall
value: 87.2434017595308
- type: similarity_accuracy
value: 68.35164835164835
- type: similarity_accuracy_threshold
value: 88.48621845245361
- type: similarity_ap
value: 73.10205506215699
- type: similarity_f1
value: 71.28712871287128
- type: similarity_f1_threshold
value: 87.00399398803711
- type: similarity_precision
value: 61.67023554603854
- type: similarity_recall
value: 84.4574780058651
task:
type: PairClassification
- dataset:
config: ru
name: MTEB XQuADRetrieval (ru)
revision: 51adfef1c1287aab1d2d91b5bead9bcfb9c68583
split: validation
type: google/xquad
metrics:
- type: main_score
value: 95.705
- type: map_at_1
value: 90.802
- type: map_at_10
value: 94.427
- type: map_at_100
value: 94.451
- type: map_at_1000
value: 94.451
- type: map_at_20
value: 94.446
- type: map_at_3
value: 94.121
- type: map_at_5
value: 94.34
- type: mrr_at_1
value: 90.80168776371308
- type: mrr_at_10
value: 94.42659567343111
- type: mrr_at_100
value: 94.45099347521871
- type: mrr_at_1000
value: 94.45099347521871
- type: mrr_at_20
value: 94.44574530017569
- type: mrr_at_3
value: 94.12095639943743
- type: mrr_at_5
value: 94.34036568213786
- type: nauc_map_at_1000_diff1
value: 87.40573202946949
- type: nauc_map_at_1000_max
value: 65.56220344468791
- type: nauc_map_at_1000_std
value: 8.865583291735863
- type: nauc_map_at_100_diff1
value: 87.40573202946949
- type: nauc_map_at_100_max
value: 65.56220344468791
- type: nauc_map_at_100_std
value: 8.865583291735863
- type: nauc_map_at_10_diff1
value: 87.43657080570291
- type: nauc_map_at_10_max
value: 65.71295628534446
- type: nauc_map_at_10_std
value: 9.055399339099655
- type: nauc_map_at_1_diff1
value: 88.08395824560428
- type: nauc_map_at_1_max
value: 62.92813192908893
- type: nauc_map_at_1_std
value: 6.738987385482432
- type: nauc_map_at_20_diff1
value: 87.40979818966589
- type: nauc_map_at_20_max
value: 65.59474346926105
- type: nauc_map_at_20_std
value: 8.944420599300914
- type: nauc_map_at_3_diff1
value: 86.97771892161035
- type: nauc_map_at_3_max
value: 66.14330030122467
- type: nauc_map_at_3_std
value: 8.62516327793521
- type: nauc_map_at_5_diff1
value: 87.30273362211798
- type: nauc_map_at_5_max
value: 66.1522476584607
- type: nauc_map_at_5_std
value: 9.780940862679724
- type: nauc_mrr_at_1000_diff1
value: 87.40573202946949
- type: nauc_mrr_at_1000_max
value: 65.56220344468791
- type: nauc_mrr_at_1000_std
value: 8.865583291735863
- type: nauc_mrr_at_100_diff1
value: 87.40573202946949
- type: nauc_mrr_at_100_max
value: 65.56220344468791
- type: nauc_mrr_at_100_std
value: 8.865583291735863
- type: nauc_mrr_at_10_diff1
value: 87.43657080570291
- type: nauc_mrr_at_10_max
value: 65.71295628534446
- type: nauc_mrr_at_10_std
value: 9.055399339099655
- type: nauc_mrr_at_1_diff1
value: 88.08395824560428
- type: nauc_mrr_at_1_max
value: 62.92813192908893
- type: nauc_mrr_at_1_std
value: 6.738987385482432
- type: nauc_mrr_at_20_diff1
value: 87.40979818966589
- type: nauc_mrr_at_20_max
value: 65.59474346926105
- type: nauc_mrr_at_20_std
value: 8.944420599300914
- type: nauc_mrr_at_3_diff1
value: 86.97771892161035
- type: nauc_mrr_at_3_max
value: 66.14330030122467
- type: nauc_mrr_at_3_std
value: 8.62516327793521
- type: nauc_mrr_at_5_diff1
value: 87.30273362211798
- type: nauc_mrr_at_5_max
value: 66.1522476584607
- type: nauc_mrr_at_5_std
value: 9.780940862679724
- type: nauc_ndcg_at_1000_diff1
value: 87.37823158814116
- type: nauc_ndcg_at_1000_max
value: 66.00874244792789
- type: nauc_ndcg_at_1000_std
value: 9.479929342875067
- type: nauc_ndcg_at_100_diff1
value: 87.37823158814116
- type: nauc_ndcg_at_100_max
value: 66.00874244792789
- type: nauc_ndcg_at_100_std
value: 9.479929342875067
- type: nauc_ndcg_at_10_diff1
value: 87.54508467181488
- type: nauc_ndcg_at_10_max
value: 66.88756470312894
- type: nauc_ndcg_at_10_std
value: 10.812624405397022
- type: nauc_ndcg_at_1_diff1
value: 88.08395824560428
- type: nauc_ndcg_at_1_max
value: 62.92813192908893
- type: nauc_ndcg_at_1_std
value: 6.738987385482432
- type: nauc_ndcg_at_20_diff1
value: 87.42097894104597
- type: nauc_ndcg_at_20_max
value: 66.37031898778943
- type: nauc_ndcg_at_20_std
value: 10.34862538094813
- type: nauc_ndcg_at_3_diff1
value: 86.50039907157999
- type: nauc_ndcg_at_3_max
value: 67.97798288917929
- type: nauc_ndcg_at_3_std
value: 10.162410286746852
- type: nauc_ndcg_at_5_diff1
value: 87.13322094568531
- type: nauc_ndcg_at_5_max
value: 68.08576118683821
- type: nauc_ndcg_at_5_std
value: 12.639637379592855
- type: nauc_precision_at_1000_diff1
value: 100.0
- type: nauc_precision_at_1000_max
value: 100.0
- type: nauc_precision_at_1000_std
value: 100.0
- type: nauc_precision_at_100_diff1
value: 100.0
- type: nauc_precision_at_100_max
value: 100.0
- type: nauc_precision_at_100_std
value: 100.0
- type: nauc_precision_at_10_diff1
value: 93.46711505595813
- type: nauc_precision_at_10_max
value: 100.0
- type: nauc_precision_at_10_std
value: 65.42573557179935
- type: nauc_precision_at_1_diff1
value: 88.08395824560428
- type: nauc_precision_at_1_max
value: 62.92813192908893
- type: nauc_precision_at_1_std
value: 6.738987385482432
- type: nauc_precision_at_20_diff1
value: 91.28948674127133
- type: nauc_precision_at_20_max
value: 100.0
- type: nauc_precision_at_20_std
value: 90.74278258632364
- type: nauc_precision_at_3_diff1
value: 82.64606115071832
- type: nauc_precision_at_3_max
value: 83.26201582412921
- type: nauc_precision_at_3_std
value: 23.334013491433762
- type: nauc_precision_at_5_diff1
value: 85.0867539350284
- type: nauc_precision_at_5_max
value: 96.57011448655484
- type: nauc_precision_at_5_std
value: 56.46869543426768
- type: nauc_recall_at_1000_diff1
value: .nan
- type: nauc_recall_at_1000_max
value: .nan
- type: nauc_recall_at_1000_std
value: .nan
- type: nauc_recall_at_100_diff1
value: .nan
- type: nauc_recall_at_100_max
value: .nan
- type: nauc_recall_at_100_std
value: .nan
- type: nauc_recall_at_10_diff1
value: 93.46711505595623
- type: nauc_recall_at_10_max
value: 100.0
- type: nauc_recall_at_10_std
value: 65.42573557180279
- type: nauc_recall_at_1_diff1
value: 88.08395824560428
- type: nauc_recall_at_1_max
value: 62.92813192908893
- type: nauc_recall_at_1_std
value: 6.738987385482432
- type: nauc_recall_at_20_diff1
value: 91.28948674127474
- type: nauc_recall_at_20_max
value: 100.0
- type: nauc_recall_at_20_std
value: 90.74278258632704
- type: nauc_recall_at_3_diff1
value: 82.64606115071967
- type: nauc_recall_at_3_max
value: 83.26201582413023
- type: nauc_recall_at_3_std
value: 23.334013491434007
- type: nauc_recall_at_5_diff1
value: 85.08675393502854
- type: nauc_recall_at_5_max
value: 96.57011448655487
- type: nauc_recall_at_5_std
value: 56.46869543426658
- type: ndcg_at_1
value: 90.802
- type: ndcg_at_10
value: 95.705
- type: ndcg_at_100
value: 95.816
- type: ndcg_at_1000
value: 95.816
- type: ndcg_at_20
value: 95.771
- type: ndcg_at_3
value: 95.11699999999999
- type: ndcg_at_5
value: 95.506
- type: precision_at_1
value: 90.802
- type: precision_at_10
value: 9.949
- type: precision_at_100
value: 1.0
- type: precision_at_1000
value: 0.1
- type: precision_at_20
value: 4.987
- type: precision_at_3
value: 32.658
- type: precision_at_5
value: 19.781000000000002
- type: recall_at_1
value: 90.802
- type: recall_at_10
value: 99.494
- type: recall_at_100
value: 100.0
- type: recall_at_1000
value: 100.0
- type: recall_at_20
value: 99.747
- type: recall_at_3
value: 97.975
- type: recall_at_5
value: 98.90299999999999
task:
type: Retrieval
tags:
- mteb
- Sentence Transformers
- sentence-similarity
- sentence-transformers
---
## Multilingual-E5-small
[Multilingual E5 Text Embeddings: A Technical Report](https://arxiv.org/pdf/2402.05672).
Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, Furu Wei, arXiv 2024
This model has 12 layers and the embedding size is 384.
## Usage
Below is an example to encode queries and passages from the MS-MARCO passage ranking dataset.
```python
import torch.nn.functional as F
from torch import Tensor
from transformers import AutoTokenizer, AutoModel
def average_pool(last_hidden_states: Tensor,
attention_mask: Tensor) -> Tensor:
last_hidden = last_hidden_states.masked_fill(~attention_mask[..., None].bool(), 0.0)
return last_hidden.sum(dim=1) / attention_mask.sum(dim=1)[..., None]
# Each input text should start with "query: " or "passage: ", even for non-English texts.
# For tasks other than retrieval, you can simply use the "query: " prefix.
input_texts = ['query: how much protein should a female eat',
'query: 南瓜的家常做法',
"passage: As a general guideline, the CDC's average requirement of protein for women ages 19 to 70 is 46 grams per day. But, as you can see from this chart, you'll need to increase that if you're expecting or training for a marathon. Check out the chart below to see how much protein you should be eating each day.",
"passage: 1.清炒南瓜丝 原料:嫩南瓜半个 调料:葱、盐、白糖、鸡精 做法: 1、南瓜用刀薄薄的削去表面一层皮,用勺子刮去瓤 2、擦成细丝(没有擦菜板就用刀慢慢切成细丝) 3、锅烧热放油,入葱花煸出香味 4、入南瓜丝快速翻炒一分钟左右,放盐、一点白糖和鸡精调味出锅 2.香葱炒南瓜 原料:南瓜1只 调料:香葱、蒜末、橄榄油、盐 做法: 1、将南瓜去皮,切成片 2、油锅8成热后,将蒜末放入爆香 3、爆香后,将南瓜片放入,翻炒 4、在翻炒的同时,可以不时地往锅里加水,但不要太多 5、放入盐,炒匀 6、南瓜差不多软和绵了之后,就可以关火 7、撒入香葱,即可出锅"]
tokenizer = AutoTokenizer.from_pretrained('intfloat/multilingual-e5-small')
model = AutoModel.from_pretrained('intfloat/multilingual-e5-small')
# Tokenize the input texts
batch_dict = tokenizer(input_texts, max_length=512, padding=True, truncation=True, return_tensors='pt')
outputs = model(**batch_dict)
embeddings = average_pool(outputs.last_hidden_state, batch_dict['attention_mask'])
# normalize embeddings
embeddings = F.normalize(embeddings, p=2, dim=1)
scores = (embeddings[:2] @ embeddings[2:].T) * 100
print(scores.tolist())
```
## Supported Languages
This model is initialized from [microsoft/Multilingual-MiniLM-L12-H384](https://huggingface.co/microsoft/Multilingual-MiniLM-L12-H384)
and continually trained on a mixture of multilingual datasets.
It supports 100 languages from xlm-roberta,
but low-resource languages may see performance degradation.
## Training Details
**Initialization**: [microsoft/Multilingual-MiniLM-L12-H384](https://huggingface.co/microsoft/Multilingual-MiniLM-L12-H384)
**First stage**: contrastive pre-training with weak supervision
| Dataset | Weak supervision | # of text pairs |
|--------------------------------------------------------------------------------------------------------|---------------------------------------|-----------------|
| Filtered [mC4](https://huggingface.co/datasets/mc4) | (title, page content) | 1B |
| [CC News](https://huggingface.co/datasets/intfloat/multilingual_cc_news) | (title, news content) | 400M |
| [NLLB](https://huggingface.co/datasets/allenai/nllb) | translation pairs | 2.4B |
| [Wikipedia](https://huggingface.co/datasets/intfloat/wikipedia) | (hierarchical section title, passage) | 150M |
| Filtered [Reddit](https://www.reddit.com/) | (comment, response) | 800M |
| [S2ORC](https://github.com/allenai/s2orc) | (title, abstract) and citation pairs | 100M |
| [Stackexchange](https://stackexchange.com/) | (question, answer) | 50M |
| [xP3](https://huggingface.co/datasets/bigscience/xP3) | (input prompt, response) | 80M |
| [Miscellaneous unsupervised SBERT data](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) | - | 10M |
**Second stage**: supervised fine-tuning
| Dataset | Language | # of text pairs |
|----------------------------------------------------------------------------------------|--------------|-----------------|
| [MS MARCO](https://microsoft.github.io/msmarco/) | English | 500k |
| [NQ](https://github.com/facebookresearch/DPR) | English | 70k |
| [Trivia QA](https://github.com/facebookresearch/DPR) | English | 60k |
| [NLI from SimCSE](https://github.com/princeton-nlp/SimCSE) | English | <300k |
| [ELI5](https://huggingface.co/datasets/eli5) | English | 500k |
| [DuReader Retrieval](https://github.com/baidu/DuReader/tree/master/DuReader-Retrieval) | Chinese | 86k |
| [KILT Fever](https://huggingface.co/datasets/kilt_tasks) | English | 70k |
| [KILT HotpotQA](https://huggingface.co/datasets/kilt_tasks) | English | 70k |
| [SQuAD](https://huggingface.co/datasets/squad) | English | 87k |
| [Quora](https://huggingface.co/datasets/quora) | English | 150k |
| [Mr. TyDi](https://huggingface.co/datasets/castorini/mr-tydi) | 11 languages | 50k |
| [MIRACL](https://huggingface.co/datasets/miracl/miracl) | 16 languages | 40k |
For all labeled datasets, we only use its training set for fine-tuning.
For other training details, please refer to our paper at [https://arxiv.org/pdf/2402.05672](https://arxiv.org/pdf/2402.05672).
## Benchmark Results on [Mr. TyDi](https://arxiv.org/abs/2108.08787)
| Model | Avg MRR@10 | | ar | bn | en | fi | id | ja | ko | ru | sw | te | th |
|-----------------------|------------|-------|------| --- | --- | --- | --- | --- | --- | --- |------| --- | --- |
| BM25 | 33.3 | | 36.7 | 41.3 | 15.1 | 28.8 | 38.2 | 21.7 | 28.1 | 32.9 | 39.6 | 42.4 | 41.7 |
| mDPR | 16.7 | | 26.0 | 25.8 | 16.2 | 11.3 | 14.6 | 18.1 | 21.9 | 18.5 | 7.3 | 10.6 | 13.5 |
| BM25 + mDPR | 41.7 | | 49.1 | 53.5 | 28.4 | 36.5 | 45.5 | 35.5 | 36.2 | 42.7 | 40.5 | 42.0 | 49.2 |
| | |
| multilingual-e5-small | 64.4 | | 71.5 | 66.3 | 54.5 | 57.7 | 63.2 | 55.4 | 54.3 | 60.8 | 65.4 | 89.1 | 70.1 |
| multilingual-e5-base | 65.9 | | 72.3 | 65.0 | 58.5 | 60.8 | 64.9 | 56.6 | 55.8 | 62.7 | 69.0 | 86.6 | 72.7 |
| multilingual-e5-large | **70.5** | | 77.5 | 73.2 | 60.8 | 66.8 | 68.5 | 62.5 | 61.6 | 65.8 | 72.7 | 90.2 | 76.2 |
## MTEB Benchmark Evaluation
Check out [unilm/e5](https://github.com/microsoft/unilm/tree/master/e5) to reproduce evaluation results
on the [BEIR](https://arxiv.org/abs/2104.08663) and [MTEB benchmark](https://arxiv.org/abs/2210.07316).
## Support for Sentence Transformers
Below is an example for usage with sentence_transformers.
```python
from sentence_transformers import SentenceTransformer
model = SentenceTransformer('intfloat/multilingual-e5-small')
input_texts = [
'query: how much protein should a female eat',
'query: 南瓜的家常做法',
"passage: As a general guideline, the CDC's average requirement of protein for women ages 19 to 70 i s 46 grams per day. But, as you can see from this chart, you'll need to increase that if you're expecting or traini ng for a marathon. Check out the chart below to see how much protein you should be eating each day.",
"passage: 1.清炒南瓜丝 原料:嫩南瓜半个 调料:葱、盐、白糖、鸡精 做法: 1、南瓜用刀薄薄的削去表面一层皮 ,用勺子刮去瓤 2、擦成细丝(没有擦菜板就用刀慢慢切成细丝) 3、锅烧热放油,入葱花煸出香味 4、入南瓜丝快速翻炒一分钟左右, 放盐、一点白糖和鸡精调味出锅 2.香葱炒南瓜 原料:南瓜1只 调料:香葱、蒜末、橄榄油、盐 做法: 1、将南瓜去皮,切成片 2、油 锅8成热后,将蒜末放入爆香 3、爆香后,将南瓜片放入,翻炒 4、在翻炒的同时,可以不时地往锅里加水,但不要太多 5、放入盐,炒匀 6、南瓜差不多软和绵了之后,就可以关火 7、撒入香葱,即可出锅"
]
embeddings = model.encode(input_texts, normalize_embeddings=True)
```
Package requirements
`pip install sentence_transformers~=2.2.2`
Contributors: [michaelfeil](https://huggingface.co/michaelfeil)
## FAQ
**1. Do I need to add the prefix "query: " and "passage: " to input texts?**
Yes, this is how the model is trained, otherwise you will see a performance degradation.
Here are some rules of thumb:
- Use "query: " and "passage: " correspondingly for asymmetric tasks such as passage retrieval in open QA, ad-hoc information retrieval.
- Use "query: " prefix for symmetric tasks such as semantic similarity, bitext mining, paraphrase retrieval.
- Use "query: " prefix if you want to use embeddings as features, such as linear probing classification, clustering.
**2. Why are my reproduced results slightly different from reported in the model card?**
Different versions of `transformers` and `pytorch` could cause negligible but non-zero performance differences.
**3. Why does the cosine similarity scores distribute around 0.7 to 1.0?**
This is a known and expected behavior as we use a low temperature 0.01 for InfoNCE contrastive loss.
For text embedding tasks like text retrieval or semantic similarity,
what matters is the relative order of the scores instead of the absolute values,
so this should not be an issue.
## Citation
If you find our paper or models helpful, please consider cite as follows:
```
@article{wang2024multilingual,
title={Multilingual E5 Text Embeddings: A Technical Report},
author={Wang, Liang and Yang, Nan and Huang, Xiaolong and Yang, Linjun and Majumder, Rangan and Wei, Furu},
journal={arXiv preprint arXiv:2402.05672},
year={2024}
}
```
## Limitations
Long texts will be truncated to at most 512 tokens.