Update README.md

README.md

@@ -1,90 +1,81 @@
----
-library_name: sentence-transformers
-pipeline_tag: sentence-similarity
-tags:
-- sentence-transformers
-- feature-extraction
-- sentence-similarity
 
----
 
-# {MODEL_NAME}
+# [bilingual-embedding-large](https://huggingface.co/Lajavaness/bilingual-embedding-large)
 
-This is a [sentence-transformers](https://www.SBERT.net) model: It maps sentences & paragraphs to a 1024 dimensional dense vector space and can be used for tasks like clustering or semantic search.
+bilingual-embedding is the Embedding Model for bilingual language: french and english. This model is a specialized sentence-embedding trained specifically for the bilingual language, leveraging the robust capabilities of [XLM-RoBERTa](https://huggingface.co/FacebookAI/xlm-roberta-large), a pre-trained language model based on the [XLM-RoBERTa](https://huggingface.co/FacebookAI/xlm-roberta-large) architecture. The model utilizes xlm-roberta to encode english-french sentences into a 1024-dimensional vector space, facilitating a wide range of applications from semantic search to text clustering. The embeddings capture the nuanced meanings of english-french sentences, reflecting both the lexical and contextual layers of the language.
 
-<!--- Describe your model here -->
 
-## Usage (Sentence-Transformers)
+## Full Model Architecture
+```
+SentenceTransformer(
+  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BilingualModel 
+  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+  (2): Normalize()
+)
+```
+
+## Training and Fine-tuning process
+The model underwent a rigorous four-stage training and fine-tuning process, each tailored to enhance its ability to generate precise and contextually relevant sentence embeddings for the Vietnamese language. Below is an outline of these stages:
+#### Stage 1: NLI Training
+- Dataset: [(SNLI+XNLI) for english+french]
+- Method: Training using Multi-Negative Ranking Loss. This stage focused on improving the model's ability to discern and rank nuanced differences in sentence semantics.
+### Stage 3: Continued Fine-tuning for Semantic Textual Similarity on STS Benchmark
+- Dataset: [STSB-fr and en]
+- Method: Fine-tuning specifically for the semantic textual similarity benchmark using Siamese BERT-Networks configured with the 'sentence-transformers' library. 
+### Stage 4: Advanced Augmentation Fine-tuning
+- Dataset: STSB-vn with generate [silver sample from gold sample](https://www.sbert.net/examples/training/data_augmentation/README.html)
+- Method: Employed an advanced strategy using [Augmented SBERT](https://arxiv.org/abs/2010.08240) with Pair Sampling Strategies, integrating both Cross-Encoder and Bi-Encoder models. This stage further refined the embeddings by enriching the training data dynamically, enhancing the model's robustness and accuracy.
+
+
+## Usage:
 
 Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:
 
 ```
 pip install -U sentence-transformers
+pip install -q pyvi
 ```
 
 Then you can use the model like this:
 
 ```python
 from sentence_transformers import SentenceTransformer
-sentences = ["This is an example sentence", "Each sentence is converted"]
-
-model = SentenceTransformer('{MODEL_NAME}')
-embeddings = model.encode(sentences)
-print(embeddings)
-```
-
-
+from pyvi.ViTokenizer import tokenize
 
-## Evaluation Results
+sentences = ["Paris est une capitale de la France", "Paris is a capital of France"]
 
-<!--- Describe how your model was evaluated -->
+model = SentenceTransformer('Lajavaness/bilingual-embedding-large', trust_remote_code=True)
+print(embeddings)
 
-For an automated evaluation of this model, see the *Sentence Embeddings Benchmark*: [https://seb.sbert.net](https://seb.sbert.net?model_name={MODEL_NAME})
+```
 
 
-## Training
-The model was trained with the parameters:
 
-**DataLoader**:
 
-`torch.utils.data.dataloader.DataLoader` of length 719 with parameters:
-```
-{'batch_size': 16, 'sampler': 'torch.utils.data.sampler.RandomSampler', 'batch_sampler': 'torch.utils.data.sampler.BatchSampler'}
-```
 
-**Loss**:
+## Evaluation
 
-`__main__.CosineSimilarityLoss` 
-
-Parameters of the fit()-Method:
-```
-{
-    "epochs": 10,
-    "evaluation_steps": 1000,
-    "evaluator": "sentence_transformers.evaluation.EmbeddingSimilarityEvaluator.EmbeddingSimilarityEvaluator",
-    "max_grad_norm": 1,
-    "optimizer_class": "<class 'torch.optim.adamw.AdamW'>",
-    "optimizer_params": {
-        "eps": 1e-06,
-        "lr": 2e-05
-    },
-    "scheduler": "WarmupLinear",
-    "steps_per_epoch": null,
-    "warmup_steps": 719,
-    "weight_decay": 0.01
-}
-```
+TODO
 
+## Citation
 
-## Full Model Architecture
-```
-SentenceTransformer(
-  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: XLMRobertaModel 
-  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
-  (2): Normalize()
-)
-```
+    @article{conneau2019unsupervised,
+      title={Unsupervised cross-lingual representation learning at scale},
+      author={Conneau, Alexis and Khandelwal, Kartikay and Goyal, Naman and Chaudhary, Vishrav and Wenzek, Guillaume and Guzm{\'a}n, Francisco and Grave, Edouard and Ott, Myle and Zettlemoyer, Luke and Stoyanov, Veselin},
+      journal={arXiv preprint arXiv:1911.02116},
+      year={2019}
+    }
 
-## Citing & Authors
+	@article{reimers2019sentence,
+	   title={Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks},
+	   author={Nils Reimers, Iryna Gurevych},
+	   journal={https://arxiv.org/abs/1908.10084},
+	   year={2019}
+	}
 
-<!--- Describe where people can find more information -->
+    @article{thakur2020augmented,
+      title={Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Tasks},
+      author={Thakur, Nandan and Reimers, Nils and Daxenberger, Johannes and Gurevych, Iryna},
+      journal={arXiv e-prints},
+      pages={arXiv--2010},
+      year={2020}