rahulseetharaman/reranker-ettin-encoder-68m-msmarco-bce-10m

CrossEncoder based on jhu-clsp/ettin-encoder-68m

This is a Cross Encoder model finetuned from jhu-clsp/ettin-encoder-68m on the msmarco dataset using the sentence-transformers library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.

Model Details

Model Description

• Model Type: Cross Encoder
• Base model: jhu-clsp/ettin-encoder-68m
• Maximum Sequence Length: 7999 tokens
• Number of Output Labels: 1 label
• Training Dataset:
  • msmarco
• Language: en

Model Sources

• Documentation: Sentence Transformers Documentation
• Documentation: Cross Encoder Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Cross Encoders on Hugging Face

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import CrossEncoder

# Download from the 🤗 Hub
model = CrossEncoder("rahulseetharaman/reranker-ettin-encoder-68m-msmarco-bce-10m")
# Get scores for pairs of texts
pairs = [
    ['select committees definition government', 'There are four types of congressional committees: 1  Standing committees, which continue from one Congress to the next, are probably the most important type because they consider and shape the vast majority of proposed laws. 2  Select committees are temporarily formed for specific purposes, often to study a particular issue.'],
    ['what is a perceptual map', 'Welcome to our New Castle, Pennsylvania street map page. The street map of New Castle PA that is located below is provided by Google Maps. You can grab the New Castle Pennsylvania street map and move it around to re-centre the map. You can change between standard map view, satellite map view and hybrid map view.'],
    ['what makes your skin feel cold and burn', 'When the wind blows in cold weather, you feel colder than the actual temperature because the air blows away heat from your skin faster. For instance, if the temperature is -17.8 Celsius (0 Fahrenheit) and the wind blows at 15 mph, it feels like -28.3 Celsius (-19 Fahrenheit) -- exposed skin can freeze in 30 minutes.'],
    ['average act score for university of georgia', 'A graph of UB, University at Buffalo GPA, SAT score, and ACT score admissions data for students who were accepted, rejected, and waitlisted. A graph of UB, University at Buffalo GPA, SAT score, and ACT score admissions data for students who were accepted, rejected, and waitlisted. University at Buffalo GPA, SAT and ACT Data Search the site GO'],
    ['when was the ontario, ca, post office established', 'In 1832 Jed Jackson had the contract for carrying mail from Brantford to London twice a week along the Old Stage Road. On October 6, 1835, a post office was established at Woodstock, Ontario, with Princeton following within two years. According to the Legislative Council Sessional Papers for 1846, a post office was established at Princeton on May 6, 1836 and Jeremiah Cowin was appointed postmaster on May 9, 1837. The sureties were George Beamer and Silas Martin to the amount of £200. The assistant was John Charles.'],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# Or rank different texts based on similarity to a single text
ranks = model.rank(
    'select committees definition government',
    [
        'There are four types of congressional committees: 1  Standing committees, which continue from one Congress to the next, are probably the most important type because they consider and shape the vast majority of proposed laws. 2  Select committees are temporarily formed for specific purposes, often to study a particular issue.',
        'Welcome to our New Castle, Pennsylvania street map page. The street map of New Castle PA that is located below is provided by Google Maps. You can grab the New Castle Pennsylvania street map and move it around to re-centre the map. You can change between standard map view, satellite map view and hybrid map view.',
        'When the wind blows in cold weather, you feel colder than the actual temperature because the air blows away heat from your skin faster. For instance, if the temperature is -17.8 Celsius (0 Fahrenheit) and the wind blows at 15 mph, it feels like -28.3 Celsius (-19 Fahrenheit) -- exposed skin can freeze in 30 minutes.',
        'A graph of UB, University at Buffalo GPA, SAT score, and ACT score admissions data for students who were accepted, rejected, and waitlisted. A graph of UB, University at Buffalo GPA, SAT score, and ACT score admissions data for students who were accepted, rejected, and waitlisted. University at Buffalo GPA, SAT and ACT Data Search the site GO',
        'In 1832 Jed Jackson had the contract for carrying mail from Brantford to London twice a week along the Old Stage Road. On October 6, 1835, a post office was established at Woodstock, Ontario, with Princeton following within two years. According to the Legislative Council Sessional Papers for 1846, a post office was established at Princeton on May 6, 1836 and Jeremiah Cowin was appointed postmaster on May 9, 1837. The sureties were George Beamer and Silas Martin to the amount of £200. The assistant was John Charles.',
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

Evaluation

Metrics

Cross Encoder Reranking

• Datasets: NanoMSMARCO_R100, NanoNFCorpus_R100 and NanoNQ_R100
• Evaluated with CrossEncoderRerankingEvaluator with these parameters:

  {
      "at_k": 10,
      "always_rerank_positives": true
  }
  

Metric	NanoMSMARCO_R100	NanoNFCorpus_R100	NanoNQ_R100
map	0.6464 (+0.1568)	0.3854 (+0.1245)	0.6581 (+0.2385)
mrr@10	0.6407 (+0.1632)	0.6487 (+0.1489)	0.6826 (+0.2559)
ndcg@10	0.7030 (+0.1626)	0.4564 (+0.1314)	0.7202 (+0.2195)

Cross Encoder Nano BEIR

• Dataset: NanoBEIR_R100_mean
• Evaluated with CrossEncoderNanoBEIREvaluator with these parameters:

  {
      "dataset_names": [
          "msmarco",
          "nfcorpus",
          "nq"
      ],
      "rerank_k": 100,
      "at_k": 10,
      "always_rerank_positives": true
  }
  

Metric	Value
map	0.5633 (+0.1732)
mrr@10	0.6573 (+0.1893)
ndcg@10	0.6265 (+0.1711)

Training Details

Training Dataset

msmarco

• Dataset: msmarco at 9e329ed
• Size: 9,960,000 training samples
• Columns: query, passage, and score
• Approximate statistics based on the first 1000 samples:

  	query	passage	score
  type	string	string	float
  details	min: 9 characters mean: 33.93 characters max: 110 characters	min: 80 characters mean: 348.08 characters max: 897 characters	min: 0.0 mean: 0.49 max: 1.0

• Samples:

  query	passage	score
  nap in chinese	continued... Most children from about 21 to 36 months of age still need one nap a day, which may range from one to three and a half hours long.They typically go to bed between 7 and 9 p.m. and wake up between 6 and 8 a.m. 3-6 Years Old: 10 - 12 hours per day.ontinued... Most children from about 21 to 36 months of age still need one nap a day, which may range from one to three and a half hours long.	0.0
  what abdominal organ is most frequently injured as a result of blunt trauma?	Bochdalek Hernia. Bochdalek hernia is a congenital posterolateral diaphragmatic defect that is a result of failed closure of the pleuroperitoneal ducts -- a primitive communications between the pleural and abdominal cavities -- at 8 weeks' gestation.ochdalek Hernia. Bochdalek hernia is a congenital posterolateral diaphragmatic defect that is a result of failed closure of the pleuroperitoneal ducts -- a primitive communications between the pleural and abdominal cavities -- at 8 weeks' gestation.	0.0
  where is round rock tx	Driving distance from Dallas, TX to Fort Worth, TX The total driving distance from Dallas, TX to Fort Worth, TX is 33 miles or 53 kilometers. Your trip begins in Dallas, Texas. It ends in Fort Worth, Texas. If you are planning a road trip, you might also want to calculate the total driving time from Dallas, TX to Fort Worth, TX so you can see when you'll arrive at your destination. You can also calculate the cost of driving from Dallas, TX to Fort Worth, TX based on current local fuel prices and an estimate of your car's best gas mileage.	0.0

• Loss: BinaryCrossEntropyLoss with these parameters:

  {
      "activation_fn": "torch.nn.modules.linear.Identity",
      "pos_weight": null
  }
  

Evaluation Dataset

msmarco

• Dataset: msmarco at 9e329ed
• Size: 40,000 evaluation samples
• Columns: query, passage, and score
• Approximate statistics based on the first 1000 samples:

  	query	passage	score
  type	string	string	float
  details	min: 11 characters mean: 34.1 characters max: 96 characters	min: 75 characters mean: 341.31 characters max: 938 characters	min: 0.0 mean: 0.49 max: 1.0

• Samples:

  query	passage	score
  select committees definition government	There are four types of congressional committees: 1 Standing committees, which continue from one Congress to the next, are probably the most important type because they consider and shape the vast majority of proposed laws. 2 Select committees are temporarily formed for specific purposes, often to study a particular issue.	1.0
  what is a perceptual map	Welcome to our New Castle, Pennsylvania street map page. The street map of New Castle PA that is located below is provided by Google Maps. You can grab the New Castle Pennsylvania street map and move it around to re-centre the map. You can change between standard map view, satellite map view and hybrid map view.	0.0
  what makes your skin feel cold and burn	When the wind blows in cold weather, you feel colder than the actual temperature because the air blows away heat from your skin faster. For instance, if the temperature is -17.8 Celsius (0 Fahrenheit) and the wind blows at 15 mph, it feels like -28.3 Celsius (-19 Fahrenheit) -- exposed skin can freeze in 30 minutes.	0.0

• Loss: BinaryCrossEntropyLoss with these parameters:

  {
      "activation_fn": "torch.nn.modules.linear.Identity",
      "pos_weight": null
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• learning_rate: 2e-05
• num_train_epochs: 4
• warmup_ratio: 0.1
• seed: 12
• bf16: True
• dataloader_num_workers: 4
• load_best_model_at_end: True

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 64
• per_device_eval_batch_size: 64
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 4
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 12
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 4
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss	NanoMSMARCO_R100_ndcg@10	NanoNFCorpus_R100_ndcg@10	NanoNQ_R100_ndcg@10	NanoBEIR_R100_mean_ndcg@10
-1	-1	-	-	0.0442 (-0.4962)	0.2555 (-0.0695)	0.0464 (-0.4542)	0.1154 (-0.3400)
0.0000	1	1.182	-	-	-	-	-
0.0643	10000	0.3736	0.1972	0.6483 (+0.1078)	0.3633 (+0.0383)	0.6880 (+0.1873)	0.5665 (+0.1112)
0.1285	20000	0.1761	0.1694	0.6341 (+0.0937)	0.4109 (+0.0858)	0.6761 (+0.1755)	0.5737 (+0.1184)
0.1928	30000	0.1552	0.1463	0.6792 (+0.1388)	0.3863 (+0.0612)	0.6831 (+0.1825)	0.5829 (+0.1275)
0.2570	40000	0.1388	0.1344	0.6949 (+0.1545)	0.4003 (+0.0753)	0.6905 (+0.1899)	0.5953 (+0.1399)
0.3213	50000	0.1297	0.1267	0.6536 (+0.1131)	0.3983 (+0.0733)	0.7293 (+0.2287)	0.5937 (+0.1384)
0.3855	60000	0.1202	0.1214	0.6771 (+0.1367)	0.3840 (+0.0589)	0.7159 (+0.2152)	0.5923 (+0.1370)
0.4498	70000	0.1124	0.1077	0.6880 (+0.1475)	0.3732 (+0.0482)	0.7256 (+0.2250)	0.5956 (+0.1402)
0.5141	80000	0.1037	0.1018	0.6684 (+0.1280)	0.3719 (+0.0469)	0.7001 (+0.1995)	0.5802 (+0.1248)
0.5783	90000	0.0956	0.0930	0.6783 (+0.1379)	0.3677 (+0.0427)	0.6870 (+0.1863)	0.5777 (+0.1223)
0.6426	100000	0.0883	0.0890	0.6971 (+0.1567)	0.4092 (+0.0842)	0.7094 (+0.2088)	0.6052 (+0.1499)
0.7068	110000	0.0839	0.0803	0.6883 (+0.1478)	0.4326 (+0.1075)	0.7181 (+0.2175)	0.6130 (+0.1576)
0.7711	120000	0.0792	0.0788	0.7013 (+0.1609)	0.4093 (+0.0843)	0.7151 (+0.2144)	0.6086 (+0.1532)
0.8353	130000	0.0755	0.0743	0.7030 (+0.1626)	0.4564 (+0.1314)	0.7202 (+0.2195)	0.6265 (+0.1711)
0.8996	140000	0.0724	0.0714	0.7014 (+0.1610)	0.3549 (+0.0299)	0.7596 (+0.2590)	0.6053 (+0.1499)
0.9639	150000	0.0688	0.0701	0.7293 (+0.1889)	0.3656 (+0.0405)	0.7116 (+0.2109)	0.6021 (+0.1468)
1.0281	160000	0.0593	0.0724	0.6591 (+0.1186)	0.3766 (+0.0515)	0.7127 (+0.2121)	0.5828 (+0.1274)
1.0924	170000	0.0514	0.0667	0.6384 (+0.0980)	0.3793 (+0.0542)	0.6664 (+0.1657)	0.5614 (+0.1060)
1.1566	180000	0.0499	0.0661	0.6643 (+0.1239)	0.3760 (+0.0510)	0.6899 (+0.1893)	0.5768 (+0.1214)
1.2209	190000	0.0479	0.0642	0.6505 (+0.1101)	0.3805 (+0.0555)	0.7143 (+0.2137)	0.5818 (+0.1264)
1.2851	200000	0.0475	0.0596	0.6545 (+0.1141)	0.3458 (+0.0208)	0.6959 (+0.1952)	0.5654 (+0.1100)
1.3494	210000	0.0459	0.0639	0.6590 (+0.1186)	0.3654 (+0.0403)	0.6613 (+0.1606)	0.5619 (+0.1065)
1.4137	220000	0.0454	0.0569	0.6402 (+0.0998)	0.3655 (+0.0405)	0.6819 (+0.1812)	0.5625 (+0.1072)
1.4779	230000	0.0446	0.0599	0.6474 (+0.1070)	0.3648 (+0.0398)	0.6731 (+0.1725)	0.5618 (+0.1064)
1.5422	240000	0.0429	0.0550	0.6416 (+0.1012)	0.3755 (+0.0505)	0.6865 (+0.1859)	0.5679 (+0.1125)
1.6064	250000	0.041	0.0563	0.6261 (+0.0857)	0.3717 (+0.0467)	0.6872 (+0.1865)	0.5617 (+0.1063)
1.6707	260000	0.04	0.0605	0.6550 (+0.1146)	0.3732 (+0.0482)	0.6900 (+0.1893)	0.5727 (+0.1174)
1.7349	270000	0.0391	0.0574	0.6292 (+0.0888)	0.3759 (+0.0508)	0.6686 (+0.1680)	0.5579 (+0.1025)
1.7992	280000	0.038	0.0519	0.6288 (+0.0884)	0.3703 (+0.0452)	0.6748 (+0.1741)	0.5580 (+0.1026)
1.8635	290000	0.0375	0.0501	0.6656 (+0.1251)	0.3793 (+0.0543)	0.7160 (+0.2154)	0.5870 (+0.1316)
1.9277	300000	0.0366	0.0472	0.6494 (+0.1090)	0.3657 (+0.0406)	0.7018 (+0.2012)	0.5723 (+0.1169)
1.9920	310000	0.0359	0.0461	0.6165 (+0.0761)	0.3812 (+0.0562)	0.6668 (+0.1661)	0.5548 (+0.0995)
2.0562	320000	0.0211	0.0598	0.5901 (+0.0497)	0.3751 (+0.0500)	0.6378 (+0.1372)	0.5343 (+0.0790)
2.1205	330000	0.02	0.0592	0.6219 (+0.0815)	0.3617 (+0.0367)	0.6317 (+0.1311)	0.5384 (+0.0831)
2.1847	340000	0.0198	0.0572	0.5953 (+0.0549)	0.3876 (+0.0626)	0.6670 (+0.1664)	0.5500 (+0.0946)
2.2490	350000	0.019	0.0605	0.5839 (+0.0435)	0.3703 (+0.0453)	0.6777 (+0.1770)	0.5440 (+0.0886)
2.3133	360000	0.0189	0.0507	0.6039 (+0.0634)	0.3756 (+0.0506)	0.6683 (+0.1677)	0.5493 (+0.0939)
2.3775	370000	0.0187	0.0522	0.6093 (+0.0688)	0.3797 (+0.0546)	0.6432 (+0.1426)	0.5441 (+0.0887)
2.4418	380000	0.0182	0.0505	0.5722 (+0.0318)	0.3674 (+0.0424)	0.6624 (+0.1618)	0.5340 (+0.0786)
2.5060	390000	0.0179	0.0448	0.5901 (+0.0497)	0.3671 (+0.0420)	0.6908 (+0.1902)	0.5493 (+0.0940)
2.5703	400000	0.0173	0.0474	0.6015 (+0.0611)	0.3500 (+0.0249)	0.6574 (+0.1567)	0.5363 (+0.0809)
2.6345	410000	0.017	0.0455	0.6096 (+0.0691)	0.3547 (+0.0296)	0.6508 (+0.1502)	0.5384 (+0.0830)
2.6988	420000	0.0168	0.0528	0.6019 (+0.0615)	0.3616 (+0.0365)	0.6860 (+0.1853)	0.5498 (+0.0944)
2.7631	430000	0.0166	0.0448	0.6017 (+0.0612)	0.3560 (+0.0310)	0.6612 (+0.1605)	0.5396 (+0.0843)
2.8273	440000	0.0152	0.0461	0.6410 (+0.1006)	0.3602 (+0.0351)	0.6418 (+0.1411)	0.5477 (+0.0923)
2.8916	450000	0.0147	0.0429	0.6335 (+0.0931)	0.3768 (+0.0517)	0.6639 (+0.1633)	0.5581 (+0.1027)
2.9558	460000	0.0148	0.0397	0.6411 (+0.1007)	0.3536 (+0.0286)	0.6693 (+0.1686)	0.5547 (+0.0993)
3.0201	470000	0.0113	0.0552	0.6431 (+0.1026)	0.3461 (+0.0211)	0.6450 (+0.1443)	0.5447 (+0.0893)
3.0843	480000	0.0051	0.0648	0.6334 (+0.0929)	0.3525 (+0.0274)	0.6761 (+0.1754)	0.5540 (+0.0986)
3.1486	490000	0.005	0.0602	0.6344 (+0.0939)	0.3519 (+0.0268)	0.6519 (+0.1513)	0.5460 (+0.0907)
3.2129	500000	0.005	0.0551	0.6287 (+0.0883)	0.3317 (+0.0067)	0.6450 (+0.1444)	0.5351 (+0.0798)
3.2771	510000	0.0047	0.0580	0.6232 (+0.0827)	0.3244 (-0.0007)	0.6508 (+0.1502)	0.5328 (+0.0774)
3.3414	520000	0.0044	0.0506	0.6047 (+0.0643)	0.3268 (+0.0017)	0.6256 (+0.1250)	0.5190 (+0.0637)
3.4056	530000	0.0044	0.0514	0.6170 (+0.0766)	0.3403 (+0.0153)	0.6354 (+0.1348)	0.5309 (+0.0755)
3.4699	540000	0.004	0.0576	0.6251 (+0.0846)	0.3388 (+0.0138)	0.6251 (+0.1245)	0.5297 (+0.0743)
3.5341	550000	0.0037	0.0487	0.6221 (+0.0817)	0.3442 (+0.0192)	0.6244 (+0.1237)	0.5302 (+0.0749)
3.5984	560000	0.0037	0.0494	0.6333 (+0.0929)	0.3371 (+0.0121)	0.6458 (+0.1451)	0.5387 (+0.0834)
3.6627	570000	0.0033	0.0426	0.6378 (+0.0974)	0.3403 (+0.0152)	0.6385 (+0.1379)	0.5389 (+0.0835)
3.7269	580000	0.0031	0.0454	0.6168 (+0.0763)	0.3473 (+0.0222)	0.6457 (+0.1451)	0.5366 (+0.0812)
3.7912	590000	0.0031	0.0434	0.6315 (+0.0911)	0.3421 (+0.0170)	0.6251 (+0.1244)	0.5329 (+0.0775)
3.8554	600000	0.0028	0.0427	0.6375 (+0.0971)	0.3406 (+0.0156)	0.6006 (+0.0999)	0.5262 (+0.0709)
3.9197	610000	0.0024	0.0397	0.6469 (+0.1065)	0.3448 (+0.0198)	0.5800 (+0.0793)	0.5239 (+0.0685)
3.9839	620000	0.0024	0.0390	0.6461 (+0.1057)	0.3473 (+0.0223)	0.5763 (+0.0757)	0.5232 (+0.0679)
-1	-1	-	-	0.7030 (+0.1626)	0.4564 (+0.1314)	0.7202 (+0.2195)	0.6265 (+0.1711)

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.10.18
• Sentence Transformers: 5.0.0
• Transformers: 4.56.0.dev0
• PyTorch: 2.7.1+cu126
• Accelerate: 1.9.0
• Datasets: 4.0.0
• Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}