MiMo: Unlocking the Reasoning Potential of Language Model -- From
  Pretraining to Posttraining
We present MiMo-7B, a large language model born for reasoning tasks, with
optimization across both pre-training and post-training stages. During
pre-training, we enhance the data preprocessing pipeline and employ a
three-stage data mixing strategy to strengthen the base model's reasoning
potential. MiMo-7B-Base is pre-trained on 25 trillion tokens, with additional
Multi-Token Prediction objective for enhanced performance and accelerated
inference speed. During post-training, we curate a dataset of 130K verifiable
mathematics and programming problems for reinforcement learning, integrating a
test-difficulty-driven code-reward scheme to alleviate sparse-reward issues and
employing strategic data resampling to stabilize training. Extensive
evaluations show that MiMo-7B-Base possesses exceptional reasoning potential,
outperforming even much larger 32B models. The final RL-tuned model,
MiMo-7B-RL, achieves superior performance on mathematics, code and general
reasoning tasks, surpassing the performance of OpenAI o1-mini. The model
checkpoints are available at https://github.com/xiaomimimo/MiMo.
6822b5441f775dd12f753469
Xiaomi LLM-Core Team
6822b5441f775dd12f75346b
Bingquan Xia
admin_assigned
2025-05-13T08:22:57.166Z
6348c3cda6aa28fa6313e906
/avatars/2c0f0b08b1371689ebb4df18ddf45e54.svg
Bingquan Xia
user
xiabingquan
6822b5441f775dd12f75346c
Bowen Shen
6822b5441f775dd12f75346d
Cici
6822b5441f775dd12f75346e
Dawei Zhu
claimed_verified
2025-05-13T08:11:55.974Z
64d2fce8129a210e569e0c76
/avatars/a79a832dc3a46ece1b9e542369fc4888.svg
Dawei Zhu
user
dwzhu
6822b5441f775dd12f75346f
Di Zhang
6822b5441f775dd12f753470
Gang Wang
6822b5441f775dd12f753471
Hailin Zhang
admin_assigned
2025-05-13T08:23:37.452Z
673dae4e4903a361fd571d1a
/avatars/df3ec2dfcacf0bd9b80b1a6955cd2a23.svg
Hailin Zhang
user
HugoZHL
6822b5441f775dd12f753472
Huaqiu Liu
claimed_verified
2025-05-13T09:54:22.144Z
680e9a219e529f779991be0c
/avatars/327b945649192b0881fe290298d10e23.svg
Huaqiu Liu
user
Prestonprom
6822b5441f775dd12f753473
Jiebao Xiao
6822b5441f775dd12f753474
Jinhao Dong
admin_assigned
2025-05-13T08:24:09.783Z
6470390f850a938d6c571cd8
/avatars/8d6eab491315e1938b2c2a52b44889f8.svg
Jinhao Dong
user
whatseeker
6822b5441f775dd12f753475
Liang Zhao
claimed_verified
2025-06-05T12:42:18.553Z
680caf5b11555c276d5bfa65
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/KQJTgmVZ1v3E4a9fQmbOp.png
Liang Zhao
user
zhao1iang-mimo
6822b5441f775dd12f753476
Peidian Li
6822b5441f775dd12f753477
Peng Wang
6822b5441f775dd12f753478
Shihua Yu
6822b5441f775dd12f753479
Shimao Chen
claimed_verified
2025-05-13T08:11:48.569Z
64f5216e3c9b5e07f85582e3
https://cdn-avatars.huggingface.co/v1/production/uploads/64f5216e3c9b5e07f85582e3/tWe8_9R2nNxQpPv-wa4nB.jpeg
Ezio Chen
user
Ezioii
6822b5441f775dd12f75347a
Weikun Wang
admin_assigned
2025-05-13T08:24:32.489Z
6822b86ea371047fe15248ab
/avatars/94a4ab0ceff03df2c996d4cf2ba7f435.svg
Weikun Wang
user
kunwww
6822b5441f775dd12f75347b
Wenhan Ma
admin_assigned
2025-05-13T08:24:41.645Z
6686f695840ee769597de318
https://cdn-avatars.huggingface.co/v1/production/uploads/6686f695840ee769597de318/50XZQ80Y2PElI35O0qVFc.jpeg
Wenhan Ma
user
CuteNPC
6822b5441f775dd12f75347c
Xiangwei Deng
admin_assigned
2025-05-13T08:24:49.400Z
65add7b1abf6d1ccb77b797c
/avatars/d386eb16053f65c0425e4878b0956fb2.svg
Xiangwei Deng
user
crybymyself
6822b5441f775dd12f75347d
Yi Huang
6822b5441f775dd12f75347e
Yifan Song
claimed_verified
2025-05-13T09:54:25.964Z
61d595f0bfc2372aaf42b766
https://cdn-avatars.huggingface.co/v1/production/uploads/1675437310060-61d595f0bfc2372aaf42b766.jpeg
Yifan Song
user
Solaris99
6822b5441f775dd12f75347f
Zihan Jiang
claimed_verified
2025-05-13T09:54:24.059Z
665ee916b9e194a20c4769da
/avatars/cfb3ff26979bed1a0c65a6fa318af3ad.svg
Zihan Jiang
user
Jumbo0715
6822b5441f775dd12f753480
Bowen Ye
admin_assigned
2025-05-13T08:25:12.162Z
663f2e2d9efa4d370061f6f6
/avatars/6e1391b66870f03b65c3fe346b6c37c0.svg
BoWen Ye
user
Ymiracle
6822b5441f775dd12f753481
Can Cai
6822b5441f775dd12f753482
Chenhong He
6822b5441f775dd12f753483
Dong Zhang
6822b5441f775dd12f753484
Duo Zhang
6822b5441f775dd12f753485
Guoan Wang
claimed_verified
2025-05-13T09:54:19.600Z
669e0733de95466572d98e74
/avatars/f50ed26f62f94c95dea6d78833564295.svg
Dylan Wang
user
Dylan0318
6822b5441f775dd12f753486
Hao Tian
6822b5441f775dd12f753487
Haochen Zhao
6822b5441f775dd12f753488
Heng Qu
6822b5441f775dd12f753489
Hongshen Xu
6822b5441f775dd12f75348a
Jun Shi
6822b5441f775dd12f75348b
Kainan Bao
6822b5441f775dd12f75348c
QingKai Fang
6822b5441f775dd12f75348d
Kang Zhou
6822b5441f775dd12f75348e
Kangyang Zhou
6822b5441f775dd12f75348f
Lei Li
claimed_verified
2025-05-13T08:11:58.039Z
6038d6d0612f5eef3cc05ea9
https://cdn-avatars.huggingface.co/v1/production/uploads/6038d6d0612f5eef3cc05ea9/ryhvAX5djQpD5OrIlZQ1f.jpeg
Lei Li
user
tobiaslee
6822b5441f775dd12f753490
Menghang Zhu
6822b5441f775dd12f753491
Nuo Chen
6822b5441f775dd12f753492
Qiantong Wang
admin_assigned
2025-05-13T08:26:40.387Z
668e4a80f4ab25acfe353946
/avatars/a94e5b16c2f058eb03fe322fe7d0b355.svg
qiantong wang
user
MightyAtom520
6822b5441f775dd12f753493
Shaohui Liu
6822b5441f775dd12f753494
Shicheng Li
claimed_verified
2025-06-05T12:42:20.608Z
64539588fc2b5f69e8faac76
/avatars/111441eeb0dd4d8ad2f0d3f28277952a.svg
Li Shicheng
user
lscpku
6822b5441f775dd12f753495
Shuhao Gu
admin_assigned
2025-05-13T08:26:56.436Z
642e72cec1b0f8e4e76af16d
/avatars/f900811d3c22a114c67283b646949f86.svg
shuhao gu
user
gsh33
6822b5441f775dd12f753496
Shuhuai Ren
claimed_verified
2025-05-13T08:11:51.419Z
60d2e681b8448e1785bbda06
https://cdn-avatars.huggingface.co/v1/production/uploads/1624434302056-noauth.jpeg
Shuhuai Ren
user
ShuhuaiRen
6822b5441f775dd12f753497
Shuo Liu
6822b5441f775dd12f753498
Sirui Deng
admin_assigned
2025-05-13T08:26:12.959Z
671560869c2303f00087f648
/avatars/2e48cf091346bbe48a4ffe0eb211870c.svg
Sirui Deng
user
sirui1228
6822b5441f775dd12f753499
Weiji Zhuang
6822b5441f775dd12f75349a
Weiwei Lv
6822b5441f775dd12f75349b
Wenyu Yang
6822b5441f775dd12f75349c
Xin Zhang
6822b5441f775dd12f75349d
Xing Yong
6822b5441f775dd12f75349e
Xing Zhang
6822b5441f775dd12f75349f
Xingchen Song
6822b5441f775dd12f7534a0
Xinzhe Xu
admin_assigned
2025-05-13T08:26:28.273Z
67e3cd03df8f424f1159a134
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/4F0LvQOERakQ_2Kvmed9u.png
Xinzhe Xu
user
muyun727
6822b5441f775dd12f7534a1
Xu Wang
6822b5441f775dd12f7534a2
Yihan Yan
6822b5441f775dd12f7534a3
Yu Tu
6822b5441f775dd12f7534a4
Yuanyuan Tian
6822b5441f775dd12f7534a5
Yudong Wang
6822b5441f775dd12f7534a6
Yue Yu
6822b5441f775dd12f7534a7
Zhenru Lin
6822b5441f775dd12f7534a8
Zhichao Song
6822b5441f775dd12f7534a9
Zihao Yue
claimed_verified
2025-05-13T08:11:53.954Z
6517f0df593b3af3120d242e
https://cdn-avatars.huggingface.co/v1/production/uploads/6517f0df593b3af3120d242e/gx4G616pnhw2ugEDPjwnH.jpeg
Zihao Yue
user
yuezih
MiMo: Unlocking the Reasoning Potential of Language Model -- From
  Pretraining to Posttraining
We present MiMo-7B, a large language model born for reasoning tasks, with
optimization across both pre-training and post-training stages. During
pre-training, we enhance the data preprocessing pipeline and employ a
three-stage data mixing strategy to strengthen the base model's reasoning
potential. MiMo-7B-Base is pre-trained on 25 trillion tokens, with additional
Multi-Token Prediction objective for enhanced performance and accelerated
inference speed. During post-training, we curate a dataset of 130K verifiable
mathematics and programming problems for reinforcement learning, integrating a
test-difficulty-driven code-reward scheme to alleviate sparse-reward issues and
employing strategic data resampling to stabilize training. Extensive
evaluations show that MiMo-7B-Base possesses exceptional reasoning potential,
outperforming even much larger 32B models. The final RL-tuned model,
MiMo-7B-RL, achieves superior performance on mathematics, code and general
reasoning tasks, surpassing the performance of OpenAI o1-mini. The model
checkpoints are available at https://github.com/xiaomimimo/MiMo.
MiMo-7B, a large language model optimized for reasoning tasks, enhances pre-training with data mixing and Multi-Token Prediction, and post-training with reinforcement learning on math and programming problems, achieving superior performance over larger models.
large language model pre-training data preprocessing three-stage data mixing strategy Multi-Token Prediction post-training reinforcement learning test-difficulty-driven code-reward scheme strategic data resampling extensive evaluations mathematics programming problems general reasoning tasks