Update README.md

README.md

@@ -1,3 +1,101 @@
----
-license: mit
----
+---
+license: mit
+base_model:
+- mistralai/Mistral-Small-24B-Instruct-2501
+tags:
+- symbolic-ai
+- reasoning
+- deductive-logic
+- glyph-code-logic-flow
+- mistral
+- mlx
+- gguf
+- fine-tuned
+- experimental
+---
+
+# Glyphstral-24B-v1 (Preview)
+
+## Model Description
+
+This is a **preview release (Version 1)** of a fine-tuned language model, **Glyphstral-24B-v1**, designed to understand and utilize the **Glyph Code Logic Flow (GCLF)** framework for structured, deductive symbolic reasoning.
+
+This model is based on **Mistral-Small-24b** and has been fine-tuned using **MLX** with **DoRA (Decomposed Relative Attention)** at 4-bit quantization on Apple Silicon.
+
+**Glyph Code Logic Flow (GCLF)** is a novel approach to symbolic AI aimed at enhancing reasoning and multi-dimensional thinking.  It provides a structured method for deductive reasoning using a symbolic language. You can explore the conceptual framework in detail here:
+
+[Computational-Model-for-Symbolic-Representations GitHub Repository](https://github.com/severian42/Computational-Model-for-Symbolic-Representations/tree/main)
+
+**Key Features (Version 1 - Preview):**
+
+*   **Specialized for Glyph Code Logic Flow:** Fine-tuned to interpret and process instructions based on the GCLF framework.
+*   **Deductive Reasoning Focus:**  Encourages structured, step-by-step deductive reasoning over probabilistic inference.
+*   **Symbolic Manipulation:**  Trained to understand and manipulate symbolic representations within the GCLF framework.
+*   **MLX Format:**  Currently provided in MLX format for efficient inference on Apple Silicon.
+*   **Quantization:**  Fine-tuned and quantized to 4-bit for reduced memory footprint and faster inference (using MLX DoRA).
+*   **Experimental V1 Release:** This is an initial release to showcase the potential of GCLF training. Expect ongoing development and improvements.
+
+## Intended Use
+
+This model is intended for **experimental use and research** in the following areas:
+
+*   **Exploring Symbolic AI:**  Investigating the capabilities of language models for structured symbolic reasoning.
+*   **Deductive Logic Applications:**  Building systems that require step-by-step, logically sound reasoning processes.
+*   **Glyph Code Logic Flow Development:**  Experimenting with and refining the GCLF framework.
+*   **Educational Purposes:**  Learning about symbolic AI, deductive reasoning, and structured knowledge representation.
+
+**Limitations:**
+
+*   **Version 1 - Preview:** This is an early version and may have limitations in robustness and generalization.
+*   **Specialized Domain:**  Performance is optimized for tasks related to Glyph Code Logic Flow. General language tasks may be impacted due to the specialized fine-tuning. (Further evaluation is ongoing)
+*   **Experimental Nature:** The GCLF framework itself is under development and this model reflects an early attempt to train an LLM for it.
+*   **MLX Format (Initial):** Currently primarily available in MLX format, which may limit accessibility for users outside the Apple Silicon/MLX ecosystem (GGUF quantization is in progress).
+
+## Training Data and Process
+
+*   **Base Model:** Mistral-Small-24b
+*   **Fine-tuning Method:** MLX-DoRA (Decomposed Relative Attention) at 4-bit quantization.
+*   **Training Hardware:** Apple M2 (128GB RAM)
+*   **Training Dataset:** Custom dataset of approximately 4500 examples specifically designed for Glyph Code Logic Flow. Each example was around 30,000 tokens in length, focused on detailed system instructions and GCLF tasks.
+*   **Training Tokens:** Approximately 27 million tokens from the custom GCLF dataset.
+*   **Training Duration:** 7 days (continuous 24/7 training).
+*   **Initial Experiments:**  Initial training attempts were made with Deepeek R1-Qwen-14 and QWQ-32, but Mistral-Small-24b was found to be more receptive to the GCLF framework due to potentially less conflicting pre-trained reasoning biases.
+
+## How to Use
+
+**System Instructions**
+
+## !! GGUF Quantization (Coming Soon) !!
+
+
+
+# Version 2 and Future Development
+
+Version 2 (In Development):
+
+GRPO (Gradient Ratio Policy Optimization): Utilizing GRPO for potentially more stable and effective fine-tuning.
+
+Newer Dataset: Training on an expanded and refined dataset for Glyph Code Logic Flow.
+
+GGUF Release: Aiming for a GGUF release for wider accessibility and compatibility.
+
+Improved Documentation: Comprehensive documentation and examples for using the model and understanding GCLF.
+
+Ongoing Efforts:
+
+Refining GCLF Framework: Continuously developing and improving the Glyph Code Logic Flow framework itself.
+
+Performance Evaluation: Conducting thorough evaluations of the model's performance on GCLF tasks and general language understanding.
+
+Community Feedback: Seeking feedback from the community to guide further development and improvements.
+
+---
+
+# Known Issues
+
+The custom dataset and heavy use of symbols and operators seems to have potentially altered the models tool use. I've found that it often want to use it's `[TOOL_CALLS]` function at the end of it's response (sometimes also calling out `<SPECIAL_#>` tokens at the end). I think I know where this is stemming from, so hopefully v2 can avoid this potential issue altogether. 
+
+If you are seeing the `[TOOL_CALLS]` and `<SPECIAL_>` outputs, you can set them as the EOS and it will align the model back into a more fluid conversation.
+
+
+