Update README.md

README.md

@@ -1,3 +1,161 @@
----
-license: agpl-3.0
----
+---
+license: agpl-3.0
+base_model:
+- meta-llama/Llama-3.1-8B-Instruct
+---
+
+# 🧠 Solphie-1S-Foundation-Model
+
+![Virende](https://huggingface.co/datasets/Virende/Solphie-1S-Foundation-Model-DS/resolve/main/logo.png)
+
+
+[![License](https://img.shields.io/badge/license-AGPL%20v3-blue?style=flat-square)](https://www.gnu.org/licenses/agpl-3.0.html)
+
+## **Overview**
+
+The **Solphie-1S-Foundation-Model** is a fine-tuned adaptation of Meta's LLaMA 3.1 8B model, purpose-built to deliver precise, context-aware assistance for developers navigating the **Solana ecosystem**. Engineered with state-of-the-art **instruction tuning**, this model excels at:  
+
+✅ **Answering complex Solana-related queries**  
+✅ **Generating high-quality, Solana-optimized code snippets**  
+✅ **Debugging smart contracts and dApps**  
+✅ **Explaining technical blockchain concepts with clarity and depth**  
+
+Designed to bridge AI intelligence with blockchain development, **Solphie-1S** empowers developers to build, optimize, and scale with **on-chain knowledge** at their fingertips.  
+
+**(Knowledge cut-off date: 29th January, 2025)**
+
+### 🎯 **Key Features**  
+- Fine-tuned with **developer-first instruction tuning**, optimized for Solana workflows.  
+- Efficient and lightweight via **LoRA (Low-Rank Adaptation)**, ensuring scalable fine-tuning.  
+- **Retains context across multi-turn conversations**, enabling seamless AI-assisted development.  
+- Generates **complete, executable code snippets** with practical real-world examples.  
+
+
+---
+
+## 🚀 **Model Card**
+
+| **Parameter**              | **Details**                                                                                  |
+|----------------------------|----------------------------------------------------------------------------------------------|
+| **Base Model**             | Meta LLaMa 3.1 8B                                                                            |
+| **Fine-Tuning Framework**  | HuggingFace Transformers, LoRA                                                               |
+| **Dataset Size**           | 13,593 high-quality Q&A pairs                                                               |
+| **Context Length**         | 4,096 tokens                                                                                 |
+| **Training Steps**         | 10,000                                                                                       |
+| **Learning Rate**          | 3e-4                                                                                        |
+| **Batch Size**             | 1 per GPU with gradient accumulation                                                         |
+| **Epochs**                 | 2                                                                                           |
+| **Model Size**             | 8 billion parameters (adapter size ~10 MB)                                                  |
+| **Pre-trained Tasks**      | Instruction following, Code generation, Debugging, Multi-turn Q&A                            |
+
+---
+
+## 📊 **Model Architecture**
+
+### **Training Workflow**
+The model was fine-tuned using parameter-efficient methods with **LoRA** to adapt to the Solana-specific domain. Below is a visualization of the training process:
+
+```
++---------------------------+               +-------------------------+
+|       Base Model          |  --- LoRA -->|  Fine-Tuned Adapter     |
+|    LLaMa 3.1 8B           |               | Virende-8B-Instruct        |
++---------------------------+               +-------------------------+
+```
+
+### **Dataset Sources**
+It is built over Virende-Novel-Instruct dataset, refer to [this page](https://huggingface.co/datasets/Virende/Solphie-1S-Foundation-Model-DS) for more details.
+
+
+---
+
+## 🛠️ **Installation and Usage**
+
+### **1. Installation**
+
+```bash
+pip install transformers datasets peft wandb
+```
+
+### **2. Load the Model**
+
+```python
+from transformers import LlamaForCausalLM, AutoTokenizer
+
+model_name = "Virende/Solphie-1S-Foundation-Model"
+
+model = LlamaForCausalLM.from_pretrained(model_name)
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+```
+
+### **3. Run Inference**
+
+```python
+def complete_chat(model, tokenizer, messages, max_new_tokens=128):
+    inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", return_dict=True, add_generation_prompt=True).to(model.device)
+    with torch.no_grad():
+        outputs = model.generate(**inputs, max_new_tokens=max_new_tokens)
+    return tokenizer.decode(outputs[0], skip_special_tokens=True)
+
+response = complete_chat(model, tokenizer, [
+    {"role": "system", "content": "You are Virende, a helpful assistant."},
+    {"role": "user", "content": "Explain how to interact with Raydium API for token swaps."}
+])
+print(response)
+```
+
+
+
+## 📂 **Dataset**
+
+| Split   | Count  | Description                    |
+|---------|--------|--------------------------------|
+| **Train** | 27.1k | High-quality Q&A pairs        |
+
+**Dataset Format (JSONL):**
+```json
+{
+  "question": "How to use the Helius API for transaction indexing?",
+  "answer": "To index transactions, use Helius's Webhooks API ...",
+  "chunk": "Helius API allows you to set up ..."
+}
+```
+
+---
+
+## 🔍 **Technical Insights**
+
+### **LoRA Configuration**
+- Rank: 8
+- Alpha: 32
+- Dropout: 0.01
+- Adapter Size: ~10 MB
+
+### **Optimization**
+- Mixed Precision (FP16) for faster inference.
+- Gradient Accumulation for memory efficiency.
+- Parameter-efficient tuning to preserve base model knowledge.
+
+---
+
+## 🙌 **Contributing**
+
+We welcome contributions to enhance the Solphie-1S Foundation Model. Feel free to:
+- Share your feedback on the HuggingFace Model Hub.
+
+---
+
+## 📜 **License**
+
+This model is licensed under the **GNU Affero General Public License v3.0 (AGPLv3).**
+
+---
+
+## 📞 **Community**
+
+For questions or support, reach out via:
+- **Twitter**: [SolphieAI](https://x.com/SolphieAI)
+---
+
+## 🤝 **Acknowledgments**
+
+Special thanks to the Solana ecosystem developers and the open-source community for their invaluable contributions and support.