Update README.md

README.md

@@ -1,50 +1,148 @@
 ---
 base_model: unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit
-library_name: transformers
 model_name: QuadConnect2.5-1.5B-v0.1.0b
+library_name: transformers
+pipeline_tag: text-generation
 tags:
-- generated_from_trainer
 - unsloth
 - trl
 - grpo
+- connect4
+- qwen
+- RL
 licence: license
+datasets:
+- Lyte/ConnectFour-T10
+language:
+- en
 ---
 
-# Model Card for QuadConnect2.5-1.5B-v0.1.0b
+# QuadConnect2.5-1.5B-v0.1.0b - A Strategic Connect Four AI
+
+![Connect Four Demo](https://cdn-uploads.huggingface.co/production/uploads/62f847d692950415b63c6011/QiDstnBXlVVz6dGrx3uus.png)
+
+## 🎮 Overview
+
+QuadConnect2.5-1.5B is a specialized language model trained to master the game of Connect Four. Built on Qwen 2.5 (1.5B parameter base), this model uses GRPO (Group Relative Policy Optimization) to learn the strategic intricacies of Connect Four gameplay.
+
+**Status**: Early training experiments (v0.0.9b) - Reward functions still evolving
 
-This model is a fine-tuned version of [unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit](https://huggingface.co/unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit).
-It has been trained using [TRL](https://github.com/huggingface/trl).
+## 🔍 Model Details
 
-## Quick start
+- **Developed by:** [Lyte](https://hf.co/Lyte)
+- **Model type:** Small Language Model (SLM)
+- **Language:** English
+- **Base model:** [unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit](https://huggingface.co/unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit)
+- **Training method:** [TRL](https://github.com/huggingface/trl)'s GRPO
+- **Training data:** [Lyte/ConnectFour-T10](https://huggingface.co/datasets/Lyte/ConnectFour-T10)
+
+## 🚀 Quick Start
+
+### Option 1: Using Transformers
 
 ```python
 from transformers import pipeline
 
-question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
+SYSTEM_PROMPT = """You are a master Connect Four strategist whose goal is to win while preventing your opponent from winning. The game is played on a 6x7 grid (columns a–g, rows 1–6 with 1 at the bottom) where pieces drop to the lowest available spot.
+
+Board:
+- Represented as a list of occupied cells in the format: <column><row>(<piece>), e.g., 'a1(O)'.
+- For example: 'a1(O), a2(X), b1(O)' indicates that cell a1 has an O, a2 has an X, and b1 has an O.
+- An empty board is shown as 'Empty Board'.
+- Win by connecting 4 pieces in any direction (horizontal, vertical, or diagonal).
+
+Strategy:
+1. Identify taken positions, and empty positions.
+2. Find and execute winning moves.
+3. If There isn't a winning move, then block your opponent's potential wins.
+4. Control the center and set up future moves.
+
+Respond in XML:
+<reasoning>
+Explain your thought process, focusing on your winning move, how you block your opponent, and your strategic plans.
+</reasoning>
+<move>
+Specify the column letter (a–g) for your next move.
+</move>
+"""
+
+board = {
+    "empty": "Game State:\n- You are playing as: X\n- Your previous moves: \n- Opponent's moves: \n- Current board state: Empty Board\n- Next available position per column:  \nColumn a: a1, a2, a3, a4, a5, a6  \nColumn b: b1, b2, b3, b4, b5, b6  \nColumn c: c1, c2, c3, c4, c5, c6  \nColumn d: d1, d2, d3, d4, d5, d6  \nColumn e: e1, e2, e3, e4, e5, e6  \nColumn f: f1, f2, f3, f4, f5, f6  \nColumn g: g1, g2, g3, g4, g5, g6\n\nMake your move.",
+    "one_move": "Game State:\n- You are playing as: X\n- Your previous moves: \n- Opponent's moves: b1\n- Current board state: b1(O)\n- Next available position per column:  \nColumn a: a1, a2, a3, a4, a5, a6  \nColumn b: b2, b3, b4, b5, b6  \nColumn c: c1, c2, c3, c4, c5, c6  \nColumn d: d1, d2, d3, d4, d5, d6  \nColumn e: e1, e2, e3, e4, e5, e6  \nColumn f: f1, f2, f3, f4, f5, f6  \nColumn g: g1, g2, g3, g4, g5, g6\n\nMake your move.",
+    "four_moves": "Game State:\n- You are playing as: X\n- Your previous moves: a1, a2\n- Opponent's moves: d1, a3\n- Current board state: a1(X), d1(O), a2(X), a3(O)\n- Next available position per column:  \nColumn a: a4, a5, a6  \nColumn b: b1, b2, b3, b4, b5, b6  \nColumn c: c1, c2, c3, c4, c5, c6  \nColumn d: d2, d3, d4, d5, d6  \nColumn e: e1, e2, e3, e4, e5, e6  \nColumn f: f1, f2, f3, f4, f5, f6  \nColumn g: g1, g2, g3, g4, g5, g6\n\nMake your move.",
+}
+
 generator = pipeline("text-generation", model="Lyte/QuadConnect2.5-1.5B-v0.1.0b", device="cuda")
-output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
+
+# use 'empty', 'one_move' or 'four_moves' in board['']
+output = generator([
+    {"role": "system", "content": SYSTEM_PROMPT}, 
+    {"role": "user", "content": board['empty']}
+], max_new_tokens=10245, return_full_text=False)[0]
+
 print(output["generated_text"])
 ```
 
-## Training procedure
+### Option 2: Using GGUF
 
- 
+Download the [Quantized GGUF (Q8_0)](https://huggingface.co/Lyte/QuadConnect2.5-1.5B-v0.1.0b/blob/main/unsloth.Q8_0.gguf) and use it in your favorite GGUF inference engine (e.g., LMStudio).
 
+### Option 3: Using Hugging Face Space
 
-This model was trained with GRPO, a method introduced in [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://huggingface.co/papers/2402.03300).
+Visit the [QuadConnect Space](https://huggingface.co/spaces/Lyte/QuadConnect) to interact with the model directly. You can also duplicate the space or download its code for local use.
 
-### Framework versions
+## 📊 Evaluation Results  
 
-- TRL: 0.15.2
+Model performance was evaluated on the [Lyte/ConnectFour-T10](https://huggingface.co/datasets/Lyte/ConnectFour-T10) validation split with various temperature settings.  
+**Note:** It appears 0.8 is the ideal temperature, and that reward functions for game strategy are harder to balance.
+
+### Summary Metrics Comparison  
+
+| Metric | v0.0.6b (Temp 0.6) | v0.0.8b (Temp 0.6) | v0.0.9b (Temp 0.6) | v0.0.9b (Temp 0.8) | v0.0.9b (Temp 1.0) | v0.1.0b (Temp 0.8) |
+|--------|-------------------|-------------------|-------------------|-------------------|-------------------|-------------------|
+| Total games evaluated | 5082 | 5082 | 5082 | 5082 | 5082 | 5082 |
+| Correct predictions | 518 | 394 | 516 | **713** | 677 | **809** |
+| Accuracy | 10.19% | 7.75% | 10.15% | **14.03%** | 13.32% | **15.92%** |
+| Most common move | d (41.14%) | d (67.61%) | a (38.72%) | a (31.01%) | a (26.99%) | e (20.89%) |
+| Middle column usage | 75.05% | 99.53% | 29.08% | 35.43% | 39.49% | 12.92% |
+
+### Move Distribution by Column  
+
+| Column | v0.0.6b (Temp 0.6) | v0.0.8b (Temp 0.6) | v0.0.9b (Temp 0.6) | v0.0.9b (Temp 0.8) | v0.0.9b (Temp 1.0) | v0.1.0b (Temp ?) |
+|--------|-------------------|-------------------|-------------------|-------------------|-------------------|-------------------|
+| a | 603 (19.02%) | 3 (0.12%) | 1447 (38.72%) | 1547 (31.01%) | 1351 (26.99%) | 803 (16.71%) |
+| b | 111 (3.50%) | 4 (0.16%) | 644 (17.23%) | 924 (18.52%) | 997 (19.92%) | 223 (4.64%) |
+| c | 785 (24.76%) | 463 (17.96%) | 648 (17.34%) | 1003 (20.11%) | 985 (19.68%) | 844 (17.57%) |
+| d | 1304 (41.14%) | 1743 (67.61%) | 101 (2.70%) | 202 (4.05%) | 306 (6.11%) | 621 (12.92%) |
+| e | 290 (9.15%) | 360 (13.96%) | 338 (9.04%) | 562 (11.27%) | 686 (13.70%) | **1004 (20.89%)** |
+| f | 50 (1.58%) | 3 (0.12%) | 310 (8.30%) | 408 (8.18%) | 354 (7.07%) | 459 (9.55%) |
+| g | 27 (0.85%) | 2 (0.08%) | 249 (6.66%) | 342 (6.86%) | 327 (6.53%) | 851 (17.71%) |
+
+
+## 🔧 Training Details
+
+### Data Preparation
+1. Started with [Leon-LLM/Connect-Four-Datasets-Collection](https://huggingface.co/datasets/Leon-LLM/Connect-Four-Datasets-Collection)
+2. Filtered for clean, complete entries
+3. Further filtered to include only games with 10 or fewer turns
+4. Split into train and validation sets
+5. Final dataset: [Lyte/ConnectFour-T10](https://huggingface.co/datasets/Lyte/ConnectFour-T10)
+
+### Evaluation Parameters
+- Temperature: 0.6, 0.8, 1.0 (compared)
+- Top-p: 0.95
+- Max tokens: 1024
+
+### Framework Versions
+- TRL: 0.15.1
 - Transformers: 4.49.0
-- Pytorch: 2.5.1+cu121
-- Datasets: 3.3.1
+- PyTorch: 2.5.1+cu121
+- Datasets: 3.2.0
 - Tokenizers: 0.21.0
 
-## Citations
-
-Cite GRPO as:
+## 📚 Citations
 
+For GRPO:
 ```bibtex
 @article{zhihong2024deepseekmath,
     title        = {{DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models}},
@@ -52,18 +150,16 @@ Cite GRPO as:
     year         = 2024,
     eprint       = {arXiv:2402.03300},
 }
-
 ```
 
-Cite TRL as:
-    
+For TRL:
 ```bibtex
 @misc{vonwerra2022trl,
-	title        = {{TRL: Transformer Reinforcement Learning}},
-	author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallouédec},
-	year         = 2020,
-	journal      = {GitHub repository},
-	publisher    = {GitHub},
-	howpublished = {\url{https://github.com/huggingface/trl}}
+    title        = {{TRL: Transformer Reinforcement Learning}},
+    author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallouédec},
+    year         = 2020,
+    journal      = {GitHub repository},
+    publisher    = {GitHub},
+    howpublished = {\url{https://github.com/huggingface/trl}}
 }
 ```