File size: 1,287 Bytes
4a06694 7d99929 4a06694 1515c18 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
---
base_model:
- BAAI/OpenSeek-Small-v1
license: open-mdw
---
# OpenSeek-Small-v1-SFT Documentation
## Overview
We adopt the [Octothinker](https://natural-rugby-f7c.notion.site/OctoThinker-Revisiting-Mid-Training-In-the-Era-of-RL-Scaling-1d20b810e2d680c494a9f9dad0a90d53) to build strong reasoning foundations. Our model's training consists of two phases: a mid-training stable phase on 200 billion tokens from a mathematical corpus, followed by a 20 billion token decay phase. Subsequently, we fine-tune the model on the [Infinity-Instruct](https://huggingface.co/datasets/BAAI/Infinity-Instruct/tree/main/7M_core) dataset to achieve superior instruction-following capabilities. This model is open-sourced as a baseline for future experiments, such as enhancing the reasoning capabilities of small models through reinforcement learning. The model architecture is the same as the OpenSeek-Small-v1 model.
## Evaluation
|Metric | GSM8K | MATH-500 | Minerva Math | OlympiadBench | Avg. |
|:--- | :--- | :--- | :--- | :--- | :--- |
|Pass@1 | 20.698 | 13.100 | 3.470 | 2.741 | 10.002 |
|Pass@4 | 41.768 | 19.100 | 8.415 | 4.997 | 18.570 |
|Pass@8 | 51.838 | 19.599 | 11.680 | 5.185 | 22.075 |
## License
[OpenMDW 1.0](https://github.com/OpenMDW/OpenMDW/blob/main/1.0/LICENSE.openmdw)
|