Add new SentenceTransformer model

1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+  "word_embedding_dimension": 1024,
+  "pooling_mode_cls_token": true,
+  "pooling_mode_mean_tokens": false,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false,
+  "pooling_mode_weightedmean_tokens": false,
+  "pooling_mode_lasttoken": false,
+  "include_prompt": true
+}

README.md

@@ -0,0 +1,750 @@
+---
+tags:
+- sentence-transformers
+- sentence-similarity
+- feature-extraction
+- generated_from_trainer
+- dataset_size:156
+- loss:MatryoshkaLoss
+- loss:MultipleNegativesRankingLoss
+base_model: Snowflake/snowflake-arctic-embed-l
+widget:
+- source_sentence: What is the significance of Claude Artifacts in the context of
+    LLMs and application development?
+  sentences:
+  - 'The environmental impact got much, much worse
+
+    The much bigger problem here is the enormous competitive buildout of the infrastructure
+    that is imagined to be necessary for these models in the future.
+
+    Companies like Google, Meta, Microsoft and Amazon are all spending billions of
+    dollars rolling out new datacenters, with a very material impact on the electricity
+    grid and the environment. There’s even talk of spinning up new nuclear power stations,
+    but those can take decades.
+
+    Is this infrastructure necessary? DeepSeek v3’s $6m training cost and the continued
+    crash in LLM prices might hint that it’s not. But would you want to be the big
+    tech executive that argued NOT to build out this infrastructure only to be proven
+    wrong in a few years’ time?'
+  - 'We already knew LLMs were spookily good at writing code. If you prompt them right,
+    it turns out they can build you a full interactive application using HTML, CSS
+    and JavaScript (and tools like React if you wire up some extra supporting build
+    mechanisms)—often in a single prompt.
+
+    Anthropic kicked this idea into high gear when they released Claude Artifacts,
+    a groundbreaking new feature that was initially slightly lost in the noise due
+    to being described half way through their announcement of the incredible Claude
+    3.5 Sonnet.
+
+    With Artifacts, Claude can write you an on-demand interactive application and
+    then let you use it directly inside the Claude interface.
+
+    Here’s my Extract URLs app, entirely generated by Claude:'
+  - 'This prompt-driven custom interface feature is so powerful and easy to build
+    (once you’ve figured out the gnarly details of browser sandboxing) that I expect
+    it to show up as a feature in a wide range of products in 2025.
+
+    Universal access to the best models lasted for just a few short months
+
+    For a few short months this year all three of the best available models—GPT-4o,
+    Claude 3.5 Sonnet and Gemini 1.5 Pro—were freely available to most of the world.'
+- source_sentence: What challenges are associated with using LLMs in the year of slop?
+  sentences:
+  - 'I also gave a bunch of talks and podcast appearances. I’ve started habitually
+    turning my talks into annotated presentations—here are my best from 2023:
+
+
+    Prompt injection explained, with video, slides, and a transcript
+
+    Catching up on the weird world of LLMs
+
+    Making Large Language Models work for you
+
+    Open questions for AI engineering
+
+    Embeddings: What they are and why they matter
+
+    Financial sustainability for open source projects at GitHub Universe
+
+
+    And in podcasts:
+
+
+
+    What AI can do for you on the Theory of Change
+
+
+    Working in public on Path to Citus Con
+
+
+    LLMs break the internet on the Changelog
+
+
+    Talking Large Language Models on Rooftop Ruby
+
+
+    Thoughts on the OpenAI board situation on Newsroom Robots'
+  - 'The year of slop
+
+    Synthetic training data works great
+
+    LLMs somehow got even harder to use
+
+    Knowledge is incredibly unevenly distributed
+
+    LLMs need better criticism
+
+    Everything tagged “llms” on my blog in 2024'
+  - 'The boring yet crucial secret behind good system prompts is test-driven development.
+    You don’t write down a system prompt and find ways to test it. You write down
+    tests and find a system prompt that passes them.
+
+
+    It’s become abundantly clear over the course of 2024 that writing good automated
+    evals for LLM-powered systems is the skill that’s most needed to build useful
+    applications on top of these models. If you have a strong eval suite you can adopt
+    new models faster, iterate better and build more reliable and useful product features
+    than your competition.
+
+    Vercel’s Malte Ubl:'
+- source_sentence: What features did GitHub and Mistral Chat introduce in relation
+    to the author's findings?
+  sentences:
+  - 'Except... you can run generated code to see if it’s correct. And with patterns
+    like ChatGPT Code Interpreter the LLM can execute the code itself, process the
+    error message, then rewrite it and keep trying until it works!
+
+    So hallucination is a much lesser problem for code generation than for anything
+    else. If only we had the equivalent of Code Interpreter for fact-checking natural
+    language!
+
+    How should we feel about this as software engineers?
+
+    On the one hand, this feels like a threat: who needs a programmer if ChatGPT can
+    write code for you?'
+  - 'I’ve found myself using this a lot. I noticed how much I was relying on it in
+    October and wrote Everything I built with Claude Artifacts this week, describing
+    14 little tools I had put together in a seven day period.
+
+    Since then, a whole bunch of other teams have built similar systems. GitHub announced
+    their version of this—GitHub Spark—in October. Mistral Chat added it as a feature
+    called Canvas in November.
+
+    Steve Krouse from Val Town built a version of it against Cerebras, showcasing
+    how a 2,000 token/second LLM can iterate on an application with changes visible
+    in less than a second.'
+  - 'This remains astonishing to me. I thought a model with the capabilities and output
+    quality of GPT-4 needed a datacenter class server with one or more $40,000+ GPUs.
+
+    These models take up enough of my 64GB of RAM that I don’t run them often—they
+    don’t leave much room for anything else.
+
+    The fact that they run at all is a testament to the incredible training and inference
+    performance gains that we’ve figured out over the past year. It turns out there
+    was a lot of low-hanging fruit to be harvested in terms of model efficiency. I
+    expect there’s still more to come.'
+- source_sentence: Why did the voice from the demo, named Skye, not make it to a production
+    product?
+  sentences:
+  - 'A lot of people are excited about AI agents—an infuriatingly vague term that
+    seems to be converging on “AI systems that can go away and act on your behalf”.
+    We’ve been talking about them all year, but I’ve seen few if any examples of them
+    running in production, despite lots of exciting prototypes.
+
+    I think this is because of gullibility.
+
+    Can we solve this? Honestly, I’m beginning to suspect that you can’t fully solve
+    gullibility without achieving AGI. So it may be quite a while before those agent
+    dreams can really start to come true!
+
+    Code may be the best application
+
+    Over the course of the year, it’s become increasingly clear that writing code
+    is one of the things LLMs are most capable of.'
+  - 'Embeddings: What they are and why they matter
+
+    61.7k
+
+    79.3k
+
+
+
+    Catching up on the weird world of LLMs
+
+    61.6k
+
+    85.9k
+
+
+
+    llamafile is the new best way to run an LLM on your own computer
+
+    52k
+
+    66k
+
+
+
+    Prompt injection explained, with video, slides, and a transcript
+
+    51k
+
+    61.9k
+
+
+
+    AI-enhanced development makes me more ambitious with my projects
+
+    49.6k
+
+    60.1k
+
+
+
+    Understanding GPT tokenizers
+
+    49.5k
+
+    61.1k
+
+
+
+    Exploring GPTs: ChatGPT in a trench coat?
+
+    46.4k
+
+    58.5k
+
+
+
+    Could you train a ChatGPT-beating model for $85,000 and run it in a browser?
+
+    40.5k
+
+    49.2k
+
+
+
+    How to implement Q&A against your documentation with GPT3, embeddings and Datasette
+
+    37.3k
+
+    44.9k
+
+
+
+    Lawyer cites fake cases invented by ChatGPT, judge is not amused
+
+    37.1k
+
+    47.4k'
+  - 'The May 13th announcement of GPT-4o included a demo of a brand new voice mode,
+    where the true multi-modal GPT-4o (the o is for “omni”) model could accept audio
+    input and output incredibly realistic sounding speech without needing separate
+    TTS or STT models.
+
+    The demo also sounded conspicuously similar to Scarlett Johansson... and after
+    she complained the voice from the demo, Skye, never made it to a production product.
+
+    The delay in releasing the new voice mode after the initial demo caused quite
+    a lot of confusion. I wrote about that in ChatGPT in “4o” mode is not running
+    the new features yet.'
+- source_sentence: What are some of the new features introduced in multi-modal models
+    that enhance their capabilities beyond text?
+  sentences:
+  - 'I think people who complain that LLM improvement has slowed are often missing
+    the enormous advances in these multi-modal models. Being able to run prompts against
+    images (and audio and video) is a fascinating new way to apply these models.
+
+    Voice and live camera mode are science fiction come to life
+
+    The audio and live video modes that have started to emerge deserve a special mention.
+
+    The ability to talk to ChatGPT first arrived in September 2023, but it was mostly
+    an illusion: OpenAI used their excellent Whisper speech-to-text model and a new
+    text-to-speech model (creatively named tts-1) to enable conversations with the
+    ChatGPT mobile apps, but the actual model just saw text.'
+  - 'Then in February, Meta released Llama. And a few weeks later in March, Georgi
+    Gerganov released code that got it working on a MacBook.
+
+    I wrote about how Large language models are having their Stable Diffusion moment,
+    and with hindsight that was a very good call!
+
+    This unleashed a whirlwind of innovation, which was accelerated further in July
+    when Meta released Llama 2—an improved version which, crucially, included permission
+    for commercial use.
+
+    Today there are literally thousands of LLMs that can be run locally, on all manner
+    of different devices.'
+  - '260 input tokens, 92 output tokens. Cost approximately 0.0024 cents (that’s less
+    than a 400th of a cent).
+
+    This increase in efficiency and reduction in price is my single favourite trend
+    from 2024. I want the utility of LLMs at a fraction of the energy cost and it
+    looks like that’s what we’re getting.
+
+    Multimodal vision is common, audio and video are starting to emerge
+
+    My butterfly example above illustrates another key trend from 2024: the rise of
+    multi-modal LLMs.
+
+    A year ago the single most notable example of these was GPT-4 Vision, released
+    at OpenAI’s DevDay in November 2023. Google’s multi-modal Gemini 1.0 was announced
+    on December 7th 2023 so it also (just) makes it into the 2023 window.'
+pipeline_tag: sentence-similarity
+library_name: sentence-transformers
+metrics:
+- cosine_accuracy@1
+- cosine_accuracy@3
+- cosine_accuracy@5
+- cosine_accuracy@10
+- cosine_precision@1
+- cosine_precision@3
+- cosine_precision@5
+- cosine_precision@10
+- cosine_recall@1
+- cosine_recall@3
+- cosine_recall@5
+- cosine_recall@10
+- cosine_ndcg@10
+- cosine_mrr@10
+- cosine_map@100
+model-index:
+- name: SentenceTransformer based on Snowflake/snowflake-arctic-embed-l
+  results:
+  - task:
+      type: information-retrieval
+      name: Information Retrieval
+    dataset:
+      name: Unknown
+      type: unknown
+    metrics:
+    - type: cosine_accuracy@1
+      value: 0.9166666666666666
+      name: Cosine Accuracy@1
+    - type: cosine_accuracy@3
+      value: 1.0
+      name: Cosine Accuracy@3
+    - type: cosine_accuracy@5
+      value: 1.0
+      name: Cosine Accuracy@5
+    - type: cosine_accuracy@10
+      value: 1.0
+      name: Cosine Accuracy@10
+    - type: cosine_precision@1
+      value: 0.9166666666666666
+      name: Cosine Precision@1
+    - type: cosine_precision@3
+      value: 0.3333333333333333
+      name: Cosine Precision@3
+    - type: cosine_precision@5
+      value: 0.20000000000000004
+      name: Cosine Precision@5
+    - type: cosine_precision@10
+      value: 0.10000000000000002
+      name: Cosine Precision@10
+    - type: cosine_recall@1
+      value: 0.9166666666666666
+      name: Cosine Recall@1
+    - type: cosine_recall@3
+      value: 1.0
+      name: Cosine Recall@3
+    - type: cosine_recall@5
+      value: 1.0
+      name: Cosine Recall@5
+    - type: cosine_recall@10
+      value: 1.0
+      name: Cosine Recall@10
+    - type: cosine_ndcg@10
+      value: 0.9692441461309548
+      name: Cosine Ndcg@10
+    - type: cosine_mrr@10
+      value: 0.9583333333333334
+      name: Cosine Mrr@10
+    - type: cosine_map@100
+      value: 0.9583333333333334
+      name: Cosine Map@100
+---
+
+# SentenceTransformer based on Snowflake/snowflake-arctic-embed-l
+
+This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [Snowflake/snowflake-arctic-embed-l](https://huggingface.co/Snowflake/snowflake-arctic-embed-l). It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
+
+## Model Details
+
+### Model Description
+- **Model Type:** Sentence Transformer
+- **Base model:** [Snowflake/snowflake-arctic-embed-l](https://huggingface.co/Snowflake/snowflake-arctic-embed-l) <!-- at revision d8fb21ca8d905d2832ee8b96c894d3298964346b -->
+- **Maximum Sequence Length:** 512 tokens
+- **Output Dimensionality:** 1024 dimensions
+- **Similarity Function:** Cosine Similarity
+<!-- - **Training Dataset:** Unknown -->
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
+- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
+
+### Full Model Architecture
+
+```
+SentenceTransformer(
+  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
+  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+  (2): Normalize()
+)
+```
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import SentenceTransformer
+
+# Download from the 🤗 Hub
+model = SentenceTransformer("njhaveri/legal-ft-2")
+# Run inference
+sentences = [
+    'What are some of the new features introduced in multi-modal models that enhance their capabilities beyond text?',
+    'I think people who complain that LLM improvement has slowed are often missing the enormous advances in these multi-modal models. Being able to run prompts against images (and audio and video) is a fascinating new way to apply these models.\nVoice and live camera mode are science fiction come to life\nThe audio and live video modes that have started to emerge deserve a special mention.\nThe ability to talk to ChatGPT first arrived in September 2023, but it was mostly an illusion: OpenAI used their excellent Whisper speech-to-text model and a new text-to-speech model (creatively named tts-1) to enable conversations with the ChatGPT mobile apps, but the actual model just saw text.',
+    '260 input tokens, 92 output tokens. Cost approximately 0.0024 cents (that’s less than a 400th of a cent).\nThis increase in efficiency and reduction in price is my single favourite trend from 2024. I want the utility of LLMs at a fraction of the energy cost and it looks like that’s what we’re getting.\nMultimodal vision is common, audio and video are starting to emerge\nMy butterfly example above illustrates another key trend from 2024: the rise of multi-modal LLMs.\nA year ago the single most notable example of these was GPT-4 Vision, released at OpenAI’s DevDay in November 2023. Google’s multi-modal Gemini 1.0 was announced on December 7th 2023 so it also (just) makes it into the 2023 window.',
+]
+embeddings = model.encode(sentences)
+print(embeddings.shape)
+# [3, 1024]
+
+# Get the similarity scores for the embeddings
+similarities = model.similarity(embeddings, embeddings)
+print(similarities.shape)
+# [3, 3]
+```
+
+<!--
+### Direct Usage (Transformers)
+
+<details><summary>Click to see the direct usage in Transformers</summary>
+
+</details>
+-->
+
+<!--
+### Downstream Usage (Sentence Transformers)
+
+You can finetune this model on your own dataset.
+
+<details><summary>Click to expand</summary>
+
+</details>
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+## Evaluation
+
+### Metrics
+
+#### Information Retrieval
+
+* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
+
+| Metric              | Value      |
+|:--------------------|:-----------|
+| cosine_accuracy@1   | 0.9167     |
+| cosine_accuracy@3   | 1.0        |
+| cosine_accuracy@5   | 1.0        |
+| cosine_accuracy@10  | 1.0        |
+| cosine_precision@1  | 0.9167     |
+| cosine_precision@3  | 0.3333     |
+| cosine_precision@5  | 0.2        |
+| cosine_precision@10 | 0.1        |
+| cosine_recall@1     | 0.9167     |
+| cosine_recall@3     | 1.0        |
+| cosine_recall@5     | 1.0        |
+| cosine_recall@10    | 1.0        |
+| **cosine_ndcg@10**  | **0.9692** |
+| cosine_mrr@10       | 0.9583     |
+| cosine_map@100      | 0.9583     |
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Dataset
+
+#### Unnamed Dataset
+
+* Size: 156 training samples
+* Columns: <code>sentence_0</code> and <code>sentence_1</code>
+* Approximate statistics based on the first 156 samples:
+  |         | sentence_0                                                                         | sentence_1                                                                           |
+  |:--------|:-----------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------|
+  | type    | string                                                                             | string                                                                               |
+  | details | <ul><li>min: 12 tokens</li><li>mean: 20.29 tokens</li><li>max: 31 tokens</li></ul> | <ul><li>min: 43 tokens</li><li>mean: 135.13 tokens</li><li>max: 214 tokens</li></ul> |
+* Samples:
+  | sentence_0                                                                                        | sentence_1                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
+  |:--------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>Why is it important for language models to believe the information provided to them?</code> | <code>Language Models are gullible. They “believe” what we tell them—what’s in their training data, then what’s in the fine-tuning data, then what’s in the prompt.<br>In order to be useful tools for us, we need them to believe what we feed them!<br>But it turns out a lot of the things we want to build need them not to be gullible.<br>Everyone wants an AI personal assistant. If you hired a real-world personal assistant who believed everything that anyone told them, you would quickly find that their ability to positively impact your life was severely limited.</code> |
+  | <code>What are the potential drawbacks of having a language model that is overly gullible?</code> | <code>Language Models are gullible. They “believe” what we tell them—what’s in their training data, then what’s in the fine-tuning data, then what’s in the prompt.<br>In order to be useful tools for us, we need them to believe what we feed them!<br>But it turns out a lot of the things we want to build need them not to be gullible.<br>Everyone wants an AI personal assistant. If you hired a real-world personal assistant who believed everything that anyone told them, you would quickly find that their ability to positively impact your life was severely limited.</code> |
+  | <code>What significant change occurred in LLM pricing over the past twelve months?</code>         | <code>Here’s the rest of the transcript. It’s bland and generic, but my phone can pitch bland and generic Christmas movies to Netflix now!<br>LLM prices crashed, thanks to competition and increased efficiency<br>The past twelve months have seen a dramatic collapse in the cost of running a prompt through the top tier hosted LLMs.<br>In December 2023 (here’s the Internet Archive for the OpenAI pricing page) OpenAI were charging $30/million input tokens for GPT-4, $10/mTok for the then-new GPT-4 Turbo and $1/mTok for GPT-3.5 Turbo.</code>                              |
+* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
+  ```json
+  {
+      "loss": "MultipleNegativesRankingLoss",
+      "matryoshka_dims": [
+          768,
+          512,
+          256,
+          128,
+          64
+      ],
+      "matryoshka_weights": [
+          1,
+          1,
+          1,
+          1,
+          1
+      ],
+      "n_dims_per_step": -1
+  }
+  ```
+
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+
+- `eval_strategy`: steps
+- `per_device_train_batch_size`: 10
+- `per_device_eval_batch_size`: 10
+- `num_train_epochs`: 10
+- `multi_dataset_batch_sampler`: round_robin
+
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+
+- `overwrite_output_dir`: False
+- `do_predict`: False
+- `eval_strategy`: steps
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 10
+- `per_device_eval_batch_size`: 10
+- `per_gpu_train_batch_size`: None
+- `per_gpu_eval_batch_size`: None
+- `gradient_accumulation_steps`: 1
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 5e-05
+- `weight_decay`: 0.0
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1
+- `num_train_epochs`: 10
+- `max_steps`: -1
+- `lr_scheduler_type`: linear
+- `lr_scheduler_kwargs`: {}
+- `warmup_ratio`: 0.0
+- `warmup_steps`: 0
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `save_safetensors`: True
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `no_cuda`: False
+- `use_cpu`: False
+- `use_mps_device`: False
+- `seed`: 42
+- `data_seed`: None
+- `jit_mode_eval`: False
+- `use_ipex`: False
+- `bf16`: False
+- `fp16`: False
+- `fp16_opt_level`: O1
+- `half_precision_backend`: auto
+- `bf16_full_eval`: False
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: 0
+- `ddp_backend`: None
+- `tpu_num_cores`: None
+- `tpu_metrics_debug`: False
+- `debug`: []
+- `dataloader_drop_last`: False
+- `dataloader_num_workers`: 0
+- `dataloader_prefetch_factor`: None
+- `past_index`: -1
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: False
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_min_num_params`: 0
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `fsdp_transformer_layer_cls_to_wrap`: None
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch
+- `optim_args`: None
+- `adafactor`: False
+- `group_by_length`: False
+- `length_column_name`: length
+- `ddp_find_unused_parameters`: None
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `use_legacy_prediction_loop`: False
+- `push_to_hub`: False
+- `resume_from_checkpoint`: None
+- `hub_model_id`: None
+- `hub_strategy`: every_save
+- `hub_private_repo`: None
+- `hub_always_push`: False
+- `gradient_checkpointing`: False
+- `gradient_checkpointing_kwargs`: None
+- `include_inputs_for_metrics`: False
+- `include_for_metrics`: []
+- `eval_do_concat_batches`: True
+- `fp16_backend`: auto
+- `push_to_hub_model_id`: None
+- `push_to_hub_organization`: None
+- `mp_parameters`: 
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `torchdynamo`: None
+- `ray_scope`: last
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `dispatch_batches`: None
+- `split_batches`: None
+- `include_tokens_per_second`: False
+- `include_num_input_tokens_seen`: False
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `use_liger_kernel`: False
+- `eval_use_gather_object`: False
+- `average_tokens_across_devices`: False
+- `prompts`: None
+- `batch_sampler`: batch_sampler
+- `multi_dataset_batch_sampler`: round_robin
+
+</details>
+
+### Training Logs
+| Epoch | Step | cosine_ndcg@10 |
+|:-----:|:----:|:--------------:|
+| 1.0   | 16   | 0.9692         |
+| 2.0   | 32   | 0.9692         |
+| 3.0   | 48   | 0.9692         |
+| 3.125 | 50   | 0.9692         |
+| 4.0   | 64   | 0.9692         |
+| 5.0   | 80   | 0.9692         |
+| 6.0   | 96   | 0.9692         |
+| 6.25  | 100  | 0.9692         |
+| 7.0   | 112  | 0.9692         |
+| 8.0   | 128  | 0.9692         |
+| 9.0   | 144  | 0.9692         |
+| 9.375 | 150  | 0.9692         |
+| 10.0  | 160  | 0.9692         |
+
+
+### Framework Versions
+- Python: 3.13.1
+- Sentence Transformers: 3.4.1
+- Transformers: 4.48.3
+- PyTorch: 2.6.0
+- Accelerate: 1.3.0
+- Datasets: 3.2.0
+- Tokenizers: 0.21.0
+
+## Citation
+
+### BibTeX
+
+#### Sentence Transformers
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
+    author = "Reimers, Nils and Gurevych, Iryna",
+    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
+    month = "11",
+    year = "2019",
+    publisher = "Association for Computational Linguistics",
+    url = "https://arxiv.org/abs/1908.10084",
+}
+```
+
+#### MatryoshkaLoss
+```bibtex
+@misc{kusupati2024matryoshka,
+    title={Matryoshka Representation Learning},
+    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
+    year={2024},
+    eprint={2205.13147},
+    archivePrefix={arXiv},
+    primaryClass={cs.LG}
+}
+```
+
+#### MultipleNegativesRankingLoss
+```bibtex
+@misc{henderson2017efficient,
+    title={Efficient Natural Language Response Suggestion for Smart Reply},
+    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
+    year={2017},
+    eprint={1705.00652},
+    archivePrefix={arXiv},
+    primaryClass={cs.CL}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,25 @@
+{
+  "_name_or_path": "Snowflake/snowflake-arctic-embed-l",
+  "architectures": [
+    "BertModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 1024,
+  "initializer_range": 0.02,
+  "intermediate_size": 4096,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 24,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float32",
+  "transformers_version": "4.48.3",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

config_sentence_transformers.json

@@ -0,0 +1,12 @@
+{
+  "__version__": {
+    "sentence_transformers": "3.4.1",
+    "transformers": "4.48.3",
+    "pytorch": "2.6.0"
+  },
+  "prompts": {
+    "query": "Represent this sentence for searching relevant passages: "
+  },
+  "default_prompt_name": null,
+  "similarity_fn_name": "cosine"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cb94cbef3c8277c537a63d41c9b283284383ba91dd566d32ee25fc142446f278
+size 1336413848

modules.json

@@ -0,0 +1,20 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  },
+  {
+    "idx": 2,
+    "name": "2",
+    "path": "2_Normalize",
+    "type": "sentence_transformers.models.Normalize"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 512,
+  "do_lower_case": false
+}

special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,63 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_lower_case": true,
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "max_length": 512,
+  "model_max_length": 512,
+  "pad_to_multiple_of": null,
+  "pad_token": "[PAD]",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "sep_token": "[SEP]",
+  "stride": 0,
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]"
+}