Initialized

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,274 @@
+---
+language:
+- en
+- ko
+license: cc-by-nc-4.0
+tags:
+- dnotitia
+- nlp
+- llm
+- slm
+- conversation
+- chat
+- reasoning
+- r1
+base_model:
+- microsoft/phi-4
+library_name: transformers
+pipeline_tag: text-generation
+---
+
+# DNA-R1
+
+<p align="center">
+<img src="assets/dna-r1-logo.png" width="400" style="margin: 40px auto;">
+</p>
+
+We introduce **DNA-R1**, a specialized reasoning model optimized for Korean language based on Microsoft's Phi-4. By applying large-scale reinforcement learning (RL) using the same methodology as DeepSeek-R1, we have significantly enhanced the model's Korean reasoning capabilities. This model demonstrates deep understanding of Korean text and exhibits exceptional reasoning abilities across mathematics, coding, and general reasoning tasks.
+
+<p align="center">
+<img src="assets/dna-r1-pipeline.png" width="100%" style="margin: 40px auto;">
+</p>
+
+## Training Methodology
+
+Our comprehensive training pipeline consists of three strategic stages:
+
+- **Stage 1:** Initial SFT with a large Korean non-reasoning dataset (760k examples) reused from our [DNA 1.0 8B Instruct](https://huggingface.co/dnotitia/Llama-DNA-1.0-8B-Instruct) training pipeline
+- **Stage 2:** Strategic integration of Korean reasoning patterns from DeepSeek R1 using a specialized Korean reasoning dataset (300k examples)
+- **Stage 3:** Advanced reinforcement learning with GRPO using a combined Korean/English reasoning dataset, with format, accuracy, and language consistency as rewards
+
+DNA-R1 has learned reasoning patterns specifically tailored for Korean language, and demonstrates capabilities such as self-verification, reflection, and generation of long chains-of-thought (CoT). This represents a significant milestone for the AI research community in the Korean language environment.
+
+## Model Specifications
+
+- **Developed by:** Dnotitia Inc.
+- **Supported Languages:** Korean, English
+- **Model Release Date:** Mar 4, 2025
+- **Number of Parameters:** 14B
+- **License:** CC BY-NC 4.0
+
+<div style="padding: 2px 8px; background-color: hsl(240, 100%, 50%, 0.1); border-radius: 5px">
+  <p><strong>NOTICE (Korean):</strong></p>
+  <p>본 모델은 상업적 목적으로 활용하실 수 있습니다. 상업적 이용을 원하시는 경우, 디노티시아 홈페이지의 <a href="https://www.dnotitia.com/contact/post-form">Contact us</a>를 통해 문의해 주시기 바랍니다. 간단한 협의 절차를 거쳐 상업적 활용을 승인해 드리도록 하겠습니다.</p>
+</div>
+
+## Technical Details
+
+### Multi-Stage Training Pipeline
+
+We implemented a sophisticated training approach to enhance Phi-4's Korean reasoning capabilities:
+
+1. **Initial Foundation (Stage 1):** Supervised Fine-Tuning using our extensive Korean non-reasoning dataset from the established [DNA 1.0 8B Instruct](https://huggingface.co/dnotitia/Llama-DNA-1.0-8B-Instruct) training pipeline
+2. **Reasoning Integration (Stage 2):** Specialized adaptation of DeepSeek R1's reasoning patterns with Korean-specific optimization through a meticulously curated dataset
+3. **Advanced Refinement (Stage 3):** Reinforcement learning optimization using GRPO to perfect reasoning in both Korean and English, with comprehensive reward signals for format structure, factual accuracy, and language consistency
+
+This methodical approach enables DNA-R1 to develop sophisticated chain-of-thought (CoT) reasoning for complex problem solving, resulting in a model finely calibrated for Korean language reasoning while maintaining robust general capabilities.
+
+### Performance Highlights
+
+Our Korean-specific multi-stage training pipeline significantly enhances the Phi-4 base model's understanding of Korean context, reasoning depth, and response capabilities. The model excels at:
+
+- Generating nuanced Korean chains-of-thought (CoT)
+- Performing rigorous self-verification
+- Solving multi-step complex problems
+- Maintaining cultural and linguistic context in reasoning
+- Distinguishing between deep thinking and concise answers using the `<think>` and `<answer>` tags
+
+## Evaluation Results
+
+Below, we present our evaluation results for the DNA-R1 model across math, coding, science, Korean, and general-performance benchmarks.
+Despite being only 14B in size, the DNA-R1 model demonstrates superior performance compared to many larger models across various benchmarks.
+
+<table>
+  <thead>
+    <tr>
+      <th>Benchmark</th>
+      <th>Task</th>
+      <th>DNA-R1-14B v1.0</th>
+      <th>DeepSeek-R1-Distill-Qwen-14B</th>
+      <th>DeepSeek-R1-Distill-Qwen-32B</th>
+      <th>EXAONE-3.5-32B-Instruct<th>
+      <th>QwQ-32B-Preview</th>
+      <th>gpt-4o-0513</th>
+      <th>o1-mini</th>
+      <th>o1-preview</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td>GSM8K</td>
+      <td rowspan="4">Math</td>
+      <td><b>92.49</b></td>
+      <td>88.63</td>
+      <td>82.64</td>
+      <td><u>91.9</u></td>
+      <td>82.41</td>
+      <td>-</td>
+      <td>-</td>
+      <td>-</td>
+    </tr>
+    <tr>
+      <td>Math500</td>
+      <td><u>89.4</u></td>
+      <td>88.2</td>
+      <td>87.4</td>
+      <td>-</td>
+      <td><b>92.2</b></td>
+      <td>75.8</td>
+      <td>85.6</td>
+      <td>81.4</td>
+    </tr>
+    <tr>
+      <td>AIME2024</td>
+      <td>53.3</td>
+      <td><u>69.7</u></td>
+      <td><b>72.6</b></td>
+      <td>6.67</td>
+      <td>50.0</td>
+      <td>8.6</td>
+      <td>64.0</td>
+      <td>40</td>
+    </tr>
+    <tr>
+      <td>OlympiadBench (Math, EN)</td>
+      <td><u>59.3</u></td>
+      <td>56.82</td>
+      <td>55.34</td>
+      <td>-</td>
+      <td><b>62.17</b></td>
+      <td>-</td>
+      <td>-</td>
+      <td>59.2</td>
+    </tr>
+    <tr>
+      <td>GPQA-Diamond</td>
+      <td>Science/Reasoning</td>
+      <td><u>61.11</u></td>
+      <td>59.1</td>
+      <td>58.08</td>
+      <td>33.33</td>
+      <td>52.5</td>
+      <td>46.5</td>
+      <td>60</td>
+      <td><b>75.2</b></td>
+    </tr>
+    <tr>
+      <td>LiveCodeBench</td>
+      <td>Coding</td>
+      <td>50.58</td>
+      <td>59.88</td>
+      <td><u>61.65</u></td>
+      <td>-</td>
+      <td>59.12</td>
+      <td>50.48</td>
+      <td><b>72.75</b></td>
+      <td>59.14</td>
+    </tr>
+    <tr>
+      <td>KMMLU-direct</td>
+      <td rowspan="3">Korean</td>
+      <td><u>59.9</u></td>
+      <td>50.5</td>
+      <td>58.62</td>
+      <td>-</td>
+      <td><b>62.96</b></td>
+      <td>-</td>
+      <td>-</td>
+      <td>-</td>
+    </tr>
+    <tr>
+      <td>KMMLU-hard</td>
+      <td><u>36.65</u></td>
+      <td>25.34</td>
+      <td>33.67</td>
+      <td>-</td>
+      <td><b>37.98</b></td>
+      <td>-</td>
+      <td>-</td>
+      <td>-</td>
+    </tr>
+    <tr>
+      <td>KoBEST</td>
+      <td><u>83.05</u></td>
+      <td>74.32</td>
+      <td>78.53</td>
+      <td>-</td>
+      <td><b>85.93</b></td>
+      <td>-</td>
+      <td>-</td>
+      <td>-</td>
+    </tr>
+    <tr>
+      <td>MMLU-Pro</td>
+      <td rowspan="3">General</td>
+      <td><u>57.64</u></td>
+      <td>50.55</td>
+      <td><b>59.58</b></td>
+      <td>-</td>
+      <td>46.82</td>
+      <td>-</td>
+      <td>-</td>
+      <td>-</td>
+    </tr>
+  </tbody>
+</table>
+
+- The *highest* *scores* are in **bold** form, and the *second*\-*highest* *scores* are <u>underlined</u>.
+- All benchmarks are evaluated with [lm-eval](https://github.com/EleutherAI/lm-evaluation-harness) and [skythought-eval](https://github.com/NovaSky-AI/SkyThought/tree/main/skythought/evals).
+
+## Quickstart
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
+
+tokenizer = AutoTokenizer.from_pretrained('dnotitia/DNA-R1')
+model = AutoModelForCausalLM.from_pretrained('dnotitia/DNA-R1', device_map='auto')
+streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
+
+conversation = [
+    {"role": "user", "content": """
+어려서부터 우리 집은 가난했었고
+남들 다하는 외식 몇 번 한 적이 없었고
+일터에 나가신 어머니 집에 없으면
+언제나 혼자서 끓여 먹었던 라면
+그러다 라면이 너무 지겨워서
+맛있는 것 좀 먹자고 대들었었어
+그러자 어머님이 마지못해 꺼내신
+숨겨두신 비상금으로 시켜주신
+짜장면 하나에 너무나 행복했었어
+하지만 어머님은 왠지 드시질 않았어
+어머님은 짜장면이 싫다고 하셨어
+어머님은 짜장면이 싫다고 하셨어
+야이야~야 그렇게 살아가고
+그렇게 후회하고 눈물도 흘리고
+야이야~야 그렇게 살아가고
+너무나 아프고 하지만 다시 웃고
+---
+친구가 쓴 시인데, 여기서 친구의 어머니가 짜장면이 싫다고 하신 이유는?"""},
+]
+inputs = tokenizer.apply_chat_template(conversation,
+                                       add_generation_prompt=True,
+                                       return_dict=True,
+                                       return_tensors="pt").to(model.device)
+_ = model.generate(**inputs, streamer=streamer)
+```
+
+
+## License
+
+This model is released under CC BY-NC 4.0 license. If you have any questions or commercial usage inquiries, please [Contact us](https://www.dnotitia.com/contact/post-form).
+
+## Citation
+
+If you use or discuss this model in your academic research, please cite the project to help spread awareness:
+
+```
+@misc{dnar12025,
+      title={DNA R1}, 
+      author={Jungyup Lee and Jemin Kim and Sang Park and SeungJae Lee},
+      year={2025},
+      publisher={HuggingFace},
+      url={https://huggingface.co/dnotitia/DNA-R1}
+}
+```