Update README.md

README.md

@@ -1,25 +1,80 @@
 ---
 library_name: transformers
-tags: []
+tags:
+- conversational-ai
+- fine-tuning
+- gpt2
+- causal-lm
+- chatbots
+license: apache-2.0
+model_name: ChatGPT-2.V2
+base_model:
+- MBZUAI/LaMini-GPT-774M
 ---
 
-### Inference
+# ChatGPT-2.V2 Model Card
 
-```python
+## Model Description
 
-# Load model directly
-from transformers import AutoModelForCausalLM, GPT2Tokenizer
+**ChatGPT-2.V2** is a fine-tuned version of the **lamini-gpt-774M** instruction model, optimized for conversational AI tasks. The model is trained to generate coherent, context-aware responses for interactive chatbot applications, achieving significant improvements in performance through fine-tuning on a combination of public conversational datasets and curated, domain-specific datasets. 
 
-tokenizer = GPT2Tokenizer.from_pretrained("suriya7/ChatGPT-2.V2")
-model = AutoModelForCausalLM.from_pretrained("suriya7/ChatGPT-2.V2")
-```
+This model supports a context length of up to **1024 tokens**, enabling it to handle multi-turn conversations effectively.
 
-### Chatting
+---
 
-```python
+## Fine-Tuning Process
+
+The model was fine-tuned using **public conversational datasets** and **curated datasets** specifically tailored for interactive chat scenarios. The fine-tuning process aimed to:
+
+- Enhance the model's ability to understand and respond to diverse conversational prompts.
+- Improve context retention and relevance in multi-turn interactions.
+- Achieve a balance between creativity and accuracy for engaging chatbot responses.
+
+The training process resulted in a **final loss of 1.2**, indicating strong convergence and performance.
+
+---
+
+## Key Features
+
+- **Conversational Proficiency:** Designed for real-time chat applications with context-aware responses.
+- **Fine-Tuned Context Handling:** Supports up to 1024 tokens, enabling robust multi-turn conversations.
+- **Instruction-Based Foundation:** Built on the lamini-gpt-774M instruction model, retaining its strengths in task-oriented dialogues.
+
+---
+
+## Training Details
+
+- **Base Model:** lamini-gpt-774M
+- **Fine-Tuning Framework:** Hugging Face Transformers
+- **Datasets Used:**
+  - Public conversational datasets (open-domain)
+  - Custom curated datasets for domain-specific conversations
+- **Context Length:** 1024 tokens
+- **Final Loss:** 1.2
+- **Learning Rate:** 1e-5
+- **Training Epochs:** 3
+- **fp16:** True
 
+---
+
+## Usage
+
+The model is intended for conversational AI applications, such as:
+
+- Chatbots for customer support
+- Interactive virtual assistants
+- Personalized conversational agents
+  
+### Inference Example
+
+```python
+# Load model directly
+from transformers import AutoModelForCausalLM, GPT2Tokenizer
 import torch
 
+tokenizer = GPT2Tokenizer.from_pretrained("suriya7/ChatGPT-2.V2")
+model = AutoModelForCausalLM.from_pretrained("suriya7/ChatGPT-2.V2")
+
 prompt = """
 <|im_start|>system\nYou are a helpful AI assistant named Securitron, trained by Aquilax.<|im_end|>
 """
@@ -58,43 +113,50 @@ while True:
 
     # Start generating tokens one by one
     assistant_response = ""
-    # print("Assistant: ", end="", flush=True)  # Print "Assistant:" once before streaming starts
     for _ in range(512):  # Specify a max token limit for streaming
-        # Generate the next token in the sequence
         next_token = model.generate(
             generated_ids,
             max_new_tokens=1,
             pad_token_id=50259,
             eos_token_id=50259,
             num_return_sequences=1,
-            do_sample=True,  # Use sampling for more diverse responses
-            top_k=50,        # Limit to the top-k tokens to sample from
-            temperature=0.7, # Adjust temperature for randomness
-            top_p =0.90
+            do_sample=True,
+            top_k=50,
+            temperature=0.7,
+            top_p=0.90
         )
         
-        # Add the generated token to the list
         generated_ids = torch.cat([generated_ids, next_token[:, -1:]], dim=1)
-        
-        # Decode the generated token (flatten it to a list of IDs)
-        token_id = next_token[0, -1].item()  # Extract the last token as an integer
+        token_id = next_token[0, -1].item()
         token = tokenizer.decode([token_id], skip_special_tokens=True)
-
         
-        # Append the token to the ongoing response
         assistant_response += token
-        print(token, end="", flush=True)  # Stream the token in real time
+        print(token, end="", flush=True)
 
-        # If EOS token is encountered, stop generating
         if token_id == 50259:  # EOS token
             break
 
-    print()  # Print a newline after streaming is complete
-
-    # Add the assistant's response to the conversation history
+    print()
     conversation_history.append(f"<|im_start|>{assistant_response.strip()}<|im_end|>")
-
 ```
 
+## Limitations
+While the model performs well in general chat scenarios, it may encounter challenges in:
+
+- Highly domain-specific contexts not covered during fine-tuning.
+- Very long conversations that exceed the 1024-token context limit.
+
+## Additional Disclaimer
+
+Please note that this model has not been specifically aligned using techniques such as Direct Preference Optimization (DPO) or similar methodologies. While the model has been fine-tuned to perform well in chat-based tasks, its responses are not guaranteed to reflect human-aligned preferences or ethical guidelines. Use with caution in sensitive or critical applications.
+
 
+## Citation
+If you use this model in your work, please cite as:
 
+@article{ChatGPT2.V2,
+  title={ChatGPT-2.V2: Fine-Tuned Conversational AI Model},
+  author={Aquilax Team},
+  year={2024},
+  note={https://huggingface.co/suriya7/ChatGPT-2.V2}
+}