README.md

---
language:
- en
- ko
license: cc-by-nc-4.0
tags:
- dnotitia
- nlp
- llm
- slm
- conversation
- chat
- reasoning
- r1
base_model:
- microsoft/phi-4
library_name: transformers
pipeline_tag: text-generation
---

# DNA-R1

<p align="center">
<img src="assets/dna-r1-logo.png" width="400" style="margin: 40px auto;">
</p>

We introduce **DNA-R1**, a specialized reasoning model optimized for Korean language based on Microsoft's Phi-4. By applying large-scale reinforcement learning (RL) using the same methodology as DeepSeek-R1, we have significantly enhanced the model's Korean reasoning capabilities. This model demonstrates deep understanding of Korean text and exhibits exceptional reasoning abilities across mathematics, coding, and general reasoning tasks.

<p align="center">
<img src="assets/dna-r1-pipeline.png" width="100%" style="margin: 40px auto;">
</p>

## Training Methodology

Our comprehensive training pipeline consists of three strategic stages:

- **Stage 1:** Initial SFT with a large Korean non-reasoning dataset (760k examples) reused from our [DNA 1.0 8B Instruct](https://huggingface.co/dnotitia/Llama-DNA-1.0-8B-Instruct) training pipeline
- **Stage 2:** Strategic integration of Korean reasoning patterns from DeepSeek R1 using a specialized Korean reasoning dataset (300k examples)
- **Stage 3:** Advanced reinforcement learning with GRPO using a combined Korean/English reasoning dataset, with format, accuracy, and language consistency as rewards

DNA-R1 has learned reasoning patterns specifically tailored for Korean language, and demonstrates capabilities such as self-verification, reflection, and generation of long chains-of-thought (CoT). This represents a significant milestone for the AI research community in the Korean language environment.

## Model Specifications

- **Developed by:** Dnotitia Inc.
- **Supported Languages:** Korean, English
- **Model Release Date:** Mar 4, 2025
- **Number of Parameters:** 14B
- **License:** CC BY-NC 4.0

<div style="padding: 2px 8px; background-color: hsl(240, 100%, 50%, 0.1); border-radius: 5px">
  <p><strong>NOTICE (Korean):</strong></p>
  <p>본 모델은 상업적 목적으로 활용하실 수 있습니다. 상업적 이용을 원하시는 경우, 디노티시아 홈페이지의 <a href="https://www.dnotitia.com/contact/post-form">Contact us</a>를 통해 문의해 주시기 바랍니다. 간단한 협의 절차를 거쳐 상업적 활용을 승인해 드리도록 하겠습니다.</p>
</div>

## Technical Details

### Multi-Stage Training Pipeline

We implemented a sophisticated training approach to enhance Phi-4's Korean reasoning capabilities:

1. **Initial Foundation (Stage 1):** Supervised Fine-Tuning using our extensive Korean non-reasoning dataset from the established [DNA 1.0 8B Instruct](https://huggingface.co/dnotitia/Llama-DNA-1.0-8B-Instruct) training pipeline
2. **Reasoning Integration (Stage 2):** Specialized adaptation of DeepSeek R1's reasoning patterns with Korean-specific optimization through a meticulously curated dataset
3. **Advanced Refinement (Stage 3):** Reinforcement learning optimization using GRPO to perfect reasoning in both Korean and English, with comprehensive reward signals for format structure, factual accuracy, and language consistency

This methodical approach enables DNA-R1 to develop sophisticated chain-of-thought (CoT) reasoning for complex problem solving, resulting in a model finely calibrated for Korean language reasoning while maintaining robust general capabilities.

### Performance Highlights

Our Korean-specific multi-stage training pipeline significantly enhances the Phi-4 base model's understanding of Korean context, reasoning depth, and response capabilities. The model excels at:

- Generating nuanced Korean chains-of-thought (CoT)
- Performing rigorous self-verification
- Solving multi-step complex problems
- Maintaining cultural and linguistic context in reasoning
- Distinguishing between deep thinking and concise answers using the `<think>` and `<answer>` tags

## Evaluation Results

Below, we present our evaluation results for the DNA-R1 model across math, coding, science, Korean, and general-performance benchmarks.
Despite being only 14B in size, the DNA-R1 model demonstrates superior performance compared to many larger models across various benchmarks.

<table>
  <thead>
    <tr>
      <th>Benchmark</th>
      <th>Task</th>
      <th>DNA-R1-14B v1.0</th>
      <th>DeepSeek-R1-Distill-Qwen-14B</th>
      <th>DeepSeek-R1-Distill-Qwen-32B</th>
      <th>EXAONE-3.5-32B-Instruct</th>
      <th>QwQ-32B-Preview</th>
      <th>gpt-4o-0513</th>
      <th>o1-mini</th>
      <th>o1-preview</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>GSM8K</td>
      <td rowspan="4">Math</td>
      <td><b>92.49</b></td>
      <td>88.63</td>
      <td>82.64</td>
      <td><u>91.9</u></td>
      <td>82.41</td>
      <td>-</td>
      <td>-</td>
      <td>-</td>
    </tr>
    <tr>
      <td>Math500</td>
      <td><u>89.4</u></td>
      <td>88.2</td>
      <td>87.4</td>
      <td>75.8</td>
      <td><b>92.2</b></td>
      <td>75.8</td>
      <td>85.6</td>
      <td>81.4</td>
    </tr>
    <tr>
      <td>AIME2024</td>
      <td>53.3</td>
      <td><u>69.7</u></td>
      <td><b>72.6</b></td>
      <td>6.67</td>
      <td>50.0</td>
      <td>8.6</td>
      <td>64.0</td>
      <td>40</td>
    </tr>
    <tr>
      <td>OlympiadBench (Math, EN)</td>
      <td><u>59.3</u></td>
      <td>56.82</td>
      <td>55.34</td>
      <td>38.58</td>
      <td><b>62.17</b></td>
      <td>-</td>
      <td>-</td>
      <td>59.2</td>
    </tr>
    <tr>
      <td>GPQA-Diamond</td>
      <td>Science/Reasoning</td>
      <td><u>61.11</u></td>
      <td>59.1</td>
      <td>58.08</td>
      <td>33.33</td>
      <td>52.5</td>
      <td>46.5</td>
      <td>60</td>
      <td><b>75.2</b></td>
    </tr>
    <tr>
      <td>LiveCodeBench</td>
      <td>Coding</td>
      <td>50.58</td>
      <td>59.88</td>
      <td><u>61.65</u></td>
      <td>19.8</td>
      <td>59.12</td>
      <td>50.48</td>
      <td><b>72.75</b></td>
      <td>59.14</td>
    </tr>
    <tr>
      <td>KMMLU-direct</td>
      <td rowspan="3">Korean</td>
      <td><u>59.9</u></td>
      <td>50.5</td>
      <td>58.62</td>
      <td>-</td>
      <td><b>62.96</b></td>
      <td>-</td>
      <td>-</td>
      <td>-</td>
    </tr>
    <tr>
      <td>KMMLU-hard</td>
      <td><u>36.65</u></td>
      <td>25.34</td>
      <td>33.67</td>
      <td>-</td>
      <td><b>37.98</b></td>
      <td>-</td>
      <td>-</td>
      <td>-</td>
    </tr>
    <tr>
      <td>KoBEST</td>
      <td><u>83.05</u></td>
      <td>74.32</td>
      <td>78.53</td>
      <td>-</td>
      <td><b>85.93</b></td>
      <td>-</td>
      <td>-</td>
      <td>-</td>
    </tr>
    <tr>
      <td>MMLU-Pro</td>
      <td rowspan="3">General</td>
      <td><u>57.64</u></td>
      <td>50.55</td>
      <td><b>59.58</b></td>
      <td>-</td>
      <td>46.82</td>
      <td>-</td>
      <td>-</td>
      <td>-</td>
    </tr>
  </tbody>
</table>

- The *highest* *scores* are in **bold** form, and the *second*\-*highest* *scores* are <u>underlined</u>.
- All benchmarks are evaluated with [lm-eval](https://github.com/EleutherAI/lm-evaluation-harness) and [skythought-eval](https://github.com/NovaSky-AI/SkyThought/tree/main/skythought/evals).

## Quickstart

```python
from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer

tokenizer = AutoTokenizer.from_pretrained('dnotitia/DNA-R1')
model = AutoModelForCausalLM.from_pretrained('dnotitia/DNA-R1', device_map='auto')
streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)

conversation = [
    {"role": "user", "content": """
어려서부터 우리 집은 가난했었고
남들 다하는 외식 몇 번 한 적이 없었고
일터에 나가신 어머니 집에 없으면
언제나 혼자서 끓여 먹었던 라면
그러다 라면이 너무 지겨워서
맛있는 것 좀 먹자고 대들었었어
그러자 어머님이 마지못해 꺼내신
숨겨두신 비상금으로 시켜주신
짜장면 하나에 너무나 행복했었어
하지만 어머님은 왠지 드시질 않았어
어머님은 짜장면이 싫다고 하셨어
어머님은 짜장면이 싫다고 하셨어
야이야~야 그렇게 살아가고
그렇게 후회하고 눈물도 흘리고
야이야~야 그렇게 살아가고
너무나 아프고 하지만 다시 웃고
---
친구가 쓴 시인데, 여기서 친구의 어머니가 짜장면이 싫다고 하신 이유는?"""},
]
inputs = tokenizer.apply_chat_template(conversation,
                                       add_generation_prompt=True,
                                       return_dict=True,
                                       return_tensors="pt").to(model.device)
_ = model.generate(**inputs, streamer=streamer)
```


## License

This model is released under CC BY-NC 4.0 license. If you have any questions or commercial usage inquiries, please [Contact us](https://www.dnotitia.com/contact/post-form).

## Citation

If you use or discuss this model in your academic research, please cite the project to help spread awareness:

```
@misc{dnar12025,
      title={DNA R1}, 
      author={Jungyup Lee and Jemin Kim and Sang Park and SeungJae Lee},
      year={2025},
      publisher={HuggingFace},
      url={https://huggingface.co/dnotitia/DNA-R1}
}
```