Update README.md with latest performance.

README.md

@@ -14,7 +14,7 @@ tags:
 # Kwaipilot OASIS-1.5B
 
 ## Model Details
-**Model Name**: OASIS (Optimized Augmentation Strategy for Improved code Search)
+**Model Name**: OASIS (Order-Augmented Strategy for Improved Code Search)
 
 **Introduction**
 
@@ -28,24 +28,19 @@ This model is ideal for developers and researchers engaged in enhancing **code r
 
 OASIS was trained on a synthetic dataset created through repository-level analysis, ensuring broad understanding across different coding styles and languages. It has demonstrated state-of-the-art performance on latest code search benchmarks.
 
-## Future Directions
-Kwaipilot upcoming initiatives include:
-
-- Open sourcing improved models.
-- Releasing technical reports.
-- Releasing natural language processing models.
-- ...
+Our preprint is now available [OASIS-arxiv](https://arxiv.org/abs/2503.08161).
 
 
 ## Performance
 
 |                 | Size | CoSQA | AdvTest | CSN-Py | CSN-Ja  | CSN-JS    | CSN-PHP   | CSN-Go    | CSN-Ruby  | Avg|
 |-----------------|:-----:|:------:|:---------:|:--------:|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:|
-|Openai-Embedding-Ada-002 | Unknown  | 0.4423| 0.3808  | 0.6802 | 0.7149| 0.6750| 0.6062| 0.8563| **0.7472**|0.6378|
+|OpenAI-Embedding-Ada-002 | Unknown  | 0.4423| 0.3808  | 0.6802 | 0.7149| 0.6750| 0.6062| 0.8563| 0.7472|0.6378|
+|OpenAI-Text-embedding-3-large | Unknown | 0.5538| 0.4684| 0.7084|	0.7292| 0.6813| 0.5959| 0.8764|0.7525|0.6707|
 |jina-embeddings-v2-base-code | 161M |**0.6837** |0.385    | 0.6634	| 0.6803| 0.6304| 0.5701| 0.8595| 0.7095|0.6477|
-| CodeSage-large          | 1.3B     | 0.4753| **0.5267**  | 0.7077 | 0.7021| **0.695** | 0.6133| 0.8371| 0.7192|0.6595|
+| CodeSage-large          | 1.3B     | 0.4753| 0.5267  | 0.7077 | 0.7021| 0.695 | 0.6133| 0.8371| 0.7192|0.6595|
 | CodeFuse-CGE-Small      | 3.8B     | 0.5619| 0.4639  | 0.6958 | 0.6863| 0.6564| 0.6133| 0.8637| 0.7341|0.6594|
-| OASIS-code-1.5B              | 1.5B     | 0.5532| 0.4861  | **0.7110**  | **0.7199**| 0.6727| **0.6217**| **0.8732**| 0.7333|**0.6713**|
+| OASIS-code-1.5B              | 1.5B     | 0.5577| **0.5727**  | **0.7369**  | **0.7397**| **0.6980**| **0.6384**| **0.8821**| **0.7547**|**0.6975**|
 
 ## Usage
 
@@ -96,20 +91,20 @@ code2 = """def quick_sort(arr):
         less = [x for x in arr[1:] if x <= pivot]
         greater = [x for x in arr[1:] if x > pivot]
         return quick_sort(less) + [pivot] + quick_sort(greater)"""
-model = AutoModel.from_pretrained("Kwaipilot/OASIS-code-1.3B", output_hidden_states=True)
-tokenizer = AutoTokenizer.from_pretrained("Kwaipilot/OASIS-code-1.3B")
+model = AutoModel.from_pretrained("Kwaipilot/OASIS-code-1.5B", output_hidden_states=True)
+tokenizer = AutoTokenizer.from_pretrained("Kwaipilot/OASIS-code-1.5B")
 
 # Tokenize and inference
-inputs = tokenizer([get_query_prompt(query), code1, code2], max_length=8192, padding=True, truncation=True, return_tensors='pt')
+inputs = tokenizer([get_query_prompt(query), code1, code2], max_length=1024, padding=True, truncation=True, return_tensors='pt')
 outputs = model(**inputs)
 # Last token pooling
 embeddings = last_token_pool(outputs.hidden_states[-1], inputs['attention_mask'])
 print(embeddings.shape)
-# torch.Size([3, 2048])
+# torch.Size([3, 1536])
 embeddings = F.normalize(embeddings, dim=1, p=2)
 similarity = embeddings @ embeddings.T
 print(similarity[0, 1:])
-# tensor([0.6495, 0.8036])
+# tensor([0.6895, 0.8240])
 ```
 ### Sentence Transformers
 First install the Sentence Transformers library:
@@ -120,7 +115,7 @@ Then you can load this model and run inference.
 ```python
 from sentence_transformers import SentenceTransformer
 # Download from the 🤗 Hub
-model = SentenceTransformer("Kwaipilot/OASIS-code-1.3B")#, model_kwargs={"torch_dtype": torch.bfloat16})
+model = SentenceTransformer("Kwaipilot/OASIS-code-1.5B")#, model_kwargs={"torch_dtype": torch.bfloat16})
 query = "How to do quicksort in python?"
 code1 = """def bubble_sort(arr):
     n = len(arr)
@@ -145,12 +140,12 @@ code2 = """def quick_sort(arr):
 query_embedding = model.encode([query], prompt_name="query")
 code_embeddings = model.encode([code1, code2])
 print(code_embeddings.shape)
-# (2, 2048)
+# (2, 1536)
 # Get the similarity scores for the embeddings
 print(model.similarity(query_embedding[0], code_embeddings[0]))
 print(model.similarity(query_embedding[0], code_embeddings[1]))
-# tensor([[0.6495]])
-# tensor([[0.8036]])
+# tensor([[0.6895]])
+# tensor([[0.8240]])
 ```
 ### BibTeX
 ```bibtex