Update README.md

README.md

@@ -14,6 +14,102 @@ tags:
 This model was converted to GGUF format from [`qihoo360/TinyR1-32B-Preview`](https://huggingface.co/qihoo360/TinyR1-32B-Preview) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
 Refer to the [original model card](https://huggingface.co/qihoo360/TinyR1-32B-Preview) for more details on the model.
 
+Original Model card:
+---
+license: apache-2.0
+library_name: transformers
+base_model:
+- deepseek-ai/DeepSeek-R1-Distill-Qwen-32B
+---
+
+**Model Name**: Tiny-R1-32B-Preview   
+**Title**: SuperDistillation Achieves Near-R1 Performance with Just 5% of Parameters.
+
+# Introduction
+We introduce our first-generation reasoning model, Tiny-R1-32B-Preview, which outperforms the 70B model Deepseek-R1-Distill-Llama-70B and nearly matches the full R1 model in math.
+
+
+## Evaluation 
+| Model                           | Math (AIME 2024)        | Coding (LiveCodeBench)  | Science (GPQA-Diamond) |
+| ------------------------------- | ------------------- | ----------------------- | ---------------------- |
+| Deepseek-R1-Distill-Qwen-32B    | 72.6        	| 57.2           	  | 62.1            |
+| Deepseek-R1-Distill-Llama-70B   | 70.0                  |  57.5                   | 65.2                   |
+| Deepseek-R1                     | 79.8                | 65.9                    | 71.5               |
+| Tiny-R1-32B-Preview (Ours)       | 78.1                | 61.6                    | 65.0      
+
+All scores are reported as pass@1.
+For AIME 2024, we sample 16 responses, and for GPQA-Diamond, we sample 4 responses, both using average overall accuracy for stable evaluation.
+
+## Approach
+| Model                           | Math (AIME 2024)        | Coding (LiveCodeBench)  | Science (GPQA-Diamond) |
+| ------------------------------- | ------------------- | ----------------------- | ---------------------- |
+| Math-Model (Ours)                | 73.1                | -                       | -                      |
+| Code-Model (Ours)              | -                   | 63.4                    | -                      |
+| Science-Model (Ours)             | -                   | -                       | 64.5                   |
+| Tiny-R1-32B-Preview (Ours)             | 78.1                | 61.6                    | 65.0 
+
+
+We applied supervised fine-tuning (SFT) to Deepseek-R1-Distill-Qwen-32B across three target domains—Mathematics, Code, and Science — using the [360-LLaMA-Factory](https://github.com/Qihoo360/360-LLaMA-Factory/) training framework to produce three domain-specific models. We used questions from open-source data as seeds, and used DeepSeek-R1 to generate responses for mathematics, coding, and science tasks separately, creating specialized models for each domain. Building on this, we leveraged the Mergekit tool from the Arcee team to combine multiple models, creating Tiny-R1-32B-Preview, which demonstrates strong overall performance.
+
+
+
+## Data
+
+#### 1. Math
+58.3k CoT trajectories from [open-r1/OpenR1-Math-220k](https://huggingface.co/datasets/open-r1/OpenR1-Math-220k), default subset
+
+#### 2. Coding
+19k CoT trajectories [open-thoughts/OpenThoughts-114k](https://huggingface.co/datasets/open-thoughts/OpenThoughts-114k), coding subset
+
+#### 3. Science
+We used R1 to generate 8 CoT trajectories on 7.6k seed examples, and got 60.8k CoT trajectories in total; the seed examples are as follows:  
+- 2.7k seed examples from [simplescaling/data_ablation_full59K](https://huggingface.co/datasets/simplescaling/data_ablation_full59K), science and health science subset
+- 4.9k seed examples from [open-thoughts/OpenThoughts-114k](https://huggingface.co/datasets/open-thoughts/OpenThoughts-114k), science subset
+
+
+
+## Open Source Plan
+We will publish a technical report as soon as possible and open-source our training and evaluation code, selected training data, and evaluation logs. Having benefited immensely from the open-source community, we are committed to giving back in every way we can.
+
+
+
+## Contributors
+
+*360 Team:*
+Lin Sun,
+Guangxiang Zhao,
+Xiaoqi Jian,
+Weihong Lin,
+Yongfu Zhu,
+Change Jia,
+Linglin Zhang,
+Jinzhu Wu,
+Sai-er Hu,
+Xiangzheng Zhang
+
+*PKU Team:*
+Yuhan Wu,
+Zihan Jiang,
+Wenrui Liu,
+Junting Zhou,
+Bin Cui,
+Tong Yang
+
+
+## Citation
+```
+@misc{tinyr1proj,
+      title={SuperDistillation Achieves Near-R1 Performance with Just 5% of Parameters.}, 
+      author={TinyR1 Team},
+      year={2025},
+      eprint={},
+      archivePrefix={},
+      primaryClass={},
+      url={https://huggingface.co/qihoo360/TinyR1-32B-Preview}, 
+}
+```
+
+
 ## Use with llama.cpp
 Install llama.cpp through brew (works on Mac and Linux)