Add new SentenceTransformer model

README.md

@@ -1,5 +1,5 @@
 ---
-base_model: nomic-ai/nomic-embed-text-v2-moe
+base_model: nomic-ai/nomic-embed-text-v2-moe-unsupervised
 library_name: sentence-transformers
 pipeline_tag: sentence-similarity
 tags:
@@ -8,15 +8,15 @@ tags:
 - feature-extraction
 ---
 
-# SentenceTransformer based on nomic-ai/nomic-embed-text-v2-moe
+# SentenceTransformer based on nomic-ai/nomic-embed-text-v2-moe-unsupervised
 
-This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [nomic-ai/nomic-embed-text-v2-moe](https://huggingface.co/nomic-ai/nomic-embed-text-v2-moe). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
+This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [nomic-ai/nomic-embed-text-v2-moe-unsupervised](https://huggingface.co/nomic-ai/nomic-embed-text-v2-moe-unsupervised). It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
 
 ## Model Details
 
 ### Model Description
 - **Model Type:** Sentence Transformer
-- **Base model:** [nomic-ai/nomic-embed-text-v2-moe](https://huggingface.co/nomic-ai/nomic-embed-text-v2-moe) <!-- at revision 8e109938f32da90ed146077b419bedd5cc6590b7 -->
+- **Base model:** [nomic-ai/nomic-embed-text-v2-moe-unsupervised](https://huggingface.co/nomic-ai/nomic-embed-text-v2-moe-unsupervised) <!-- at revision e48a32f5906ed18933f85467e57c1dcc02ef401b -->
 - **Maximum Sequence Length:** 512 tokens
 - **Output Dimensionality:** 768 dimensions
 - **Similarity Function:** Cosine Similarity
@@ -115,8 +115,8 @@ You can finetune this model on your own dataset.
 - Sentence Transformers: 3.3.0
 - Transformers: 4.44.2
 - PyTorch: 2.4.1+cu121
-- Accelerate: 1.0.0
-- Datasets: 2.19.0
+- Accelerate: 1.2.1
+- Datasets: 3.2.0
 - Tokenizers: 0.19.1
 
 ## Citation

config.json

@@ -1,5 +1,5 @@
 {
-  "_name_or_path": "nomic-ai/nomic-xlm-2048",
+  "_name_or_path": "nomic-ai/nomic-embed-text-v2-moe-unsupervised",
   "activation_function": "gelu",
   "add_pooling_layer": false,
   "architectures": [
@@ -55,7 +55,7 @@
   "rotary_emb_interleaved": false,
   "rotary_emb_scale_base": null,
   "rotary_scaling_factor": null,
-  "router_aux_loss_coef": 0.1,
+  "router_aux_loss_coef": 1.0,
   "scale_attn_by_inverse_layer_idx": false,
   "scale_attn_weights": true,
   "summary_activation": null,

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:097012b27af76d80af74fed4bc2ccc9091245286f776adf03ad1758a24ade9a0
+oid sha256:d28e71c4128b432588d35ff44b2ad70839cf0c639a8096d2f44cef5018b655ee
 size 1901187232