title: MiMo: Unlocking the Reasoning Potential of Language Model -- From
    Pretraining to Posttraining
summary: We present MiMo-7B, a large language model born for reasoning tasks, with
  optimization across both pre-training and post-training stages. During
  pre-training, we enhance the data preprocessing pipeline and employ a
  three-stage data mixing strategy to strengthen the base model's reasoning
  potential. MiMo-7B-Base is pre-trained on 25 trillion tokens, with additional
  Multi-Token Prediction objective for enhanced performance and accelerated
  inference speed. During post-training, we curate a dataset of 130K verifiable
  mathematics and programming problems for reinforcement learning, integrating a
  test-difficulty-driven code-reward scheme to alleviate sparse-reward issues and
  employing strategic data resampling to stabilize training. Extensive
  evaluations show that MiMo-7B-Base possesses exceptional reasoning potential,
  outperforming even much larger 32B models. The final RL-tuned model,
  MiMo-7B-RL, achieves superior performance on mathematics, code and general
  reasoning tasks, surpassing the performance of OpenAI o1-mini. The model
  checkpoints are available at https://github.com/xiaomimimo/MiMo.
paper_authors._id: 6822b5441f775dd12f753469
paper_authors.name: Xiaomi LLM-Core Team
paper_authors._id: 6822b5441f775dd12f75346b
paper_authors.name: Bingquan Xia
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-13T08:22:57.166Z
paper_authors._id: 6348c3cda6aa28fa6313e906
paper_authors.avatarUrl: /avatars/2c0f0b08b1371689ebb4df18ddf45e54.svg
paper_authors.fullname: Bingquan Xia
paper_authors.type: user
paper_authors.user: xiabingquan
paper_authors._id: 6822b5441f775dd12f75346c
paper_authors.name: Bowen Shen
paper_authors._id: 6822b5441f775dd12f75346d
paper_authors.name: Cici
paper_authors._id: 6822b5441f775dd12f75346e
paper_authors.name: Dawei Zhu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-13T08:11:55.974Z
paper_authors._id: 64d2fce8129a210e569e0c76
paper_authors.avatarUrl: /avatars/a79a832dc3a46ece1b9e542369fc4888.svg
paper_authors.fullname: Dawei Zhu
paper_authors.type: user
paper_authors.user: dwzhu
paper_authors._id: 6822b5441f775dd12f75346f
paper_authors.name: Di Zhang
paper_authors._id: 6822b5441f775dd12f753470
paper_authors.name: Gang Wang
paper_authors._id: 6822b5441f775dd12f753471
paper_authors.name: Hailin Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-13T08:23:37.452Z
paper_authors._id: 673dae4e4903a361fd571d1a
paper_authors.avatarUrl: /avatars/df3ec2dfcacf0bd9b80b1a6955cd2a23.svg
paper_authors.fullname: Hailin Zhang
paper_authors.type: user
paper_authors.user: HugoZHL
paper_authors._id: 6822b5441f775dd12f753472
paper_authors.name: Huaqiu Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-13T09:54:22.144Z
paper_authors._id: 680e9a219e529f779991be0c
paper_authors.avatarUrl: /avatars/327b945649192b0881fe290298d10e23.svg
paper_authors.fullname: Huaqiu Liu
paper_authors.type: user
paper_authors.user: Prestonprom
paper_authors._id: 6822b5441f775dd12f753473
paper_authors.name: Jiebao Xiao
paper_authors._id: 6822b5441f775dd12f753474
paper_authors.name: Jinhao Dong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-13T08:24:09.783Z
paper_authors._id: 6470390f850a938d6c571cd8
paper_authors.avatarUrl: /avatars/8d6eab491315e1938b2c2a52b44889f8.svg
paper_authors.fullname: Jinhao Dong
paper_authors.type: user
paper_authors.user: whatseeker
paper_authors._id: 6822b5441f775dd12f753475
paper_authors.name: Liang Zhao
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-05T12:42:18.553Z
paper_authors._id: 680caf5b11555c276d5bfa65
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/KQJTgmVZ1v3E4a9fQmbOp.png
paper_authors.fullname: Liang Zhao
paper_authors.type: user
paper_authors.user: zhao1iang-mimo
paper_authors._id: 6822b5441f775dd12f753476
paper_authors.name: Peidian Li
paper_authors._id: 6822b5441f775dd12f753477
paper_authors.name: Peng Wang
paper_authors._id: 6822b5441f775dd12f753478
paper_authors.name: Shihua Yu
paper_authors._id: 6822b5441f775dd12f753479
paper_authors.name: Shimao Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-13T08:11:48.569Z
paper_authors._id: 64f5216e3c9b5e07f85582e3
paper_authors.avatarUrl: /avatars/7b5303f3fb4bc223ee6217403ed5c949.svg
paper_authors.fullname: Ezio Chen
paper_authors.type: user
paper_authors.user: Ezioii
paper_authors._id: 6822b5441f775dd12f75347a
paper_authors.name: Weikun Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-13T08:24:32.489Z
paper_authors._id: 6822b86ea371047fe15248ab
paper_authors.avatarUrl: /avatars/94a4ab0ceff03df2c996d4cf2ba7f435.svg
paper_authors.fullname: Weikun Wang
paper_authors.type: user
paper_authors.user: kunwww
paper_authors._id: 6822b5441f775dd12f75347b
paper_authors.name: Wenhan Ma
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-13T08:24:41.645Z
paper_authors._id: 6686f695840ee769597de318
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6686f695840ee769597de318/50XZQ80Y2PElI35O0qVFc.jpeg
paper_authors.fullname: Wenhan Ma
paper_authors.type: user
paper_authors.user: CuteNPC
paper_authors._id: 6822b5441f775dd12f75347c
paper_authors.name: Xiangwei Deng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-13T08:24:49.400Z
paper_authors._id: 65add7b1abf6d1ccb77b797c
paper_authors.avatarUrl: /avatars/d386eb16053f65c0425e4878b0956fb2.svg
paper_authors.fullname: Xiangwei Deng
paper_authors.type: user
paper_authors.user: crybymyself
paper_authors._id: 6822b5441f775dd12f75347d
paper_authors.name: Yi Huang
paper_authors._id: 6822b5441f775dd12f75347e
paper_authors.name: Yifan Song
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-13T09:54:25.964Z
paper_authors._id: 61d595f0bfc2372aaf42b766
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1675437310060-61d595f0bfc2372aaf42b766.jpeg
paper_authors.fullname: Yifan Song
paper_authors.type: user
paper_authors.user: Solaris99
paper_authors._id: 6822b5441f775dd12f75347f
paper_authors.name: Zihan Jiang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-13T09:54:24.059Z
paper_authors._id: 665ee916b9e194a20c4769da
paper_authors.avatarUrl: /avatars/cfb3ff26979bed1a0c65a6fa318af3ad.svg
paper_authors.fullname: Zihan Jiang
paper_authors.type: user
paper_authors.user: Jumbo0715
paper_authors._id: 6822b5441f775dd12f753480
paper_authors.name: Bowen Ye
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-13T08:25:12.162Z
paper_authors._id: 663f2e2d9efa4d370061f6f6
paper_authors.avatarUrl: /avatars/6e1391b66870f03b65c3fe346b6c37c0.svg
paper_authors.fullname: BoWen Ye
paper_authors.type: user
paper_authors.user: Ymiracle
paper_authors._id: 6822b5441f775dd12f753481
paper_authors.name: Can Cai
paper_authors._id: 6822b5441f775dd12f753482
paper_authors.name: Chenhong He
paper_authors._id: 6822b5441f775dd12f753483
paper_authors.name: Dong Zhang
paper_authors._id: 6822b5441f775dd12f753484
paper_authors.name: Duo Zhang
paper_authors._id: 6822b5441f775dd12f753485
paper_authors.name: Guoan Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-13T09:54:19.600Z
paper_authors._id: 669e0733de95466572d98e74
paper_authors.avatarUrl: /avatars/f50ed26f62f94c95dea6d78833564295.svg
paper_authors.fullname: Dylan Wang
paper_authors.type: user
paper_authors.user: Dylan0318
paper_authors._id: 6822b5441f775dd12f753486
paper_authors.name: Hao Tian
paper_authors._id: 6822b5441f775dd12f753487
paper_authors.name: Haochen Zhao
paper_authors._id: 6822b5441f775dd12f753488
paper_authors.name: Heng Qu
paper_authors._id: 6822b5441f775dd12f753489
paper_authors.name: Hongshen Xu
paper_authors._id: 6822b5441f775dd12f75348a
paper_authors.name: Jun Shi
paper_authors._id: 6822b5441f775dd12f75348b
paper_authors.name: Kainan Bao
paper_authors._id: 6822b5441f775dd12f75348c
paper_authors.name: QingKai Fang
paper_authors._id: 6822b5441f775dd12f75348d
paper_authors.name: Kang Zhou
paper_authors._id: 6822b5441f775dd12f75348e
paper_authors.name: Kangyang Zhou
paper_authors._id: 6822b5441f775dd12f75348f
paper_authors.name: Lei Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-13T08:11:58.039Z
paper_authors._id: 6038d6d0612f5eef3cc05ea9
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6038d6d0612f5eef3cc05ea9/ryhvAX5djQpD5OrIlZQ1f.jpeg
paper_authors.fullname: Lei Li
paper_authors.type: user
paper_authors.user: tobiaslee
paper_authors._id: 6822b5441f775dd12f753490
paper_authors.name: Menghang Zhu
paper_authors._id: 6822b5441f775dd12f753491
paper_authors.name: Nuo Chen
paper_authors._id: 6822b5441f775dd12f753492
paper_authors.name: Qiantong Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-13T08:26:40.387Z
paper_authors._id: 668e4a80f4ab25acfe353946
paper_authors.avatarUrl: /avatars/a94e5b16c2f058eb03fe322fe7d0b355.svg
paper_authors.fullname: qiantong wang
paper_authors.type: user
paper_authors.user: MightyAtom520
paper_authors._id: 6822b5441f775dd12f753493
paper_authors.name: Shaohui Liu
paper_authors._id: 6822b5441f775dd12f753494
paper_authors.name: Shicheng Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-05T12:42:20.608Z
paper_authors._id: 64539588fc2b5f69e8faac76
paper_authors.avatarUrl: /avatars/111441eeb0dd4d8ad2f0d3f28277952a.svg
paper_authors.fullname: Li Shicheng
paper_authors.type: user
paper_authors.user: lscpku
paper_authors._id: 6822b5441f775dd12f753495
paper_authors.name: Shuhao Gu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-13T08:26:56.436Z
paper_authors._id: 642e72cec1b0f8e4e76af16d
paper_authors.avatarUrl: /avatars/f900811d3c22a114c67283b646949f86.svg
paper_authors.fullname: shuhao gu
paper_authors.type: user
paper_authors.user: gsh33
paper_authors._id: 6822b5441f775dd12f753496
paper_authors.name: Shuhuai Ren
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-13T08:11:51.419Z
paper_authors._id: 60d2e681b8448e1785bbda06
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1624434302056-noauth.jpeg
paper_authors.fullname: Shuhuai Ren
paper_authors.type: user
paper_authors.user: ShuhuaiRen
paper_authors._id: 6822b5441f775dd12f753497
paper_authors.name: Shuo Liu
paper_authors._id: 6822b5441f775dd12f753498
paper_authors.name: Sirui Deng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-13T08:26:12.959Z
paper_authors._id: 671560869c2303f00087f648
paper_authors.avatarUrl: /avatars/2e48cf091346bbe48a4ffe0eb211870c.svg
paper_authors.fullname: Sirui Deng
paper_authors.type: user
paper_authors.user: sirui1228
paper_authors._id: 6822b5441f775dd12f753499
paper_authors.name: Weiji Zhuang
paper_authors._id: 6822b5441f775dd12f75349a
paper_authors.name: Weiwei Lv
paper_authors._id: 6822b5441f775dd12f75349b
paper_authors.name: Wenyu Yang
paper_authors._id: 6822b5441f775dd12f75349c
paper_authors.name: Xin Zhang
paper_authors._id: 6822b5441f775dd12f75349d
paper_authors.name: Xing Yong
paper_authors._id: 6822b5441f775dd12f75349e
paper_authors.name: Xing Zhang
paper_authors._id: 6822b5441f775dd12f75349f
paper_authors.name: Xingchen Song
paper_authors._id: 6822b5441f775dd12f7534a0
paper_authors.name: Xinzhe Xu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-13T08:26:28.273Z
paper_authors._id: 67e3cd03df8f424f1159a134
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/4F0LvQOERakQ_2Kvmed9u.png
paper_authors.fullname: Xinzhe Xu
paper_authors.type: user
paper_authors.user: muyun727
paper_authors._id: 6822b5441f775dd12f7534a1
paper_authors.name: Xu Wang
paper_authors._id: 6822b5441f775dd12f7534a2
paper_authors.name: Yihan Yan
paper_authors._id: 6822b5441f775dd12f7534a3
paper_authors.name: Yu Tu
paper_authors._id: 6822b5441f775dd12f7534a4
paper_authors.name: Yuanyuan Tian
paper_authors._id: 6822b5441f775dd12f7534a5
paper_authors.name: Yudong Wang
paper_authors._id: 6822b5441f775dd12f7534a6
paper_authors.name: Yue Yu
paper_authors._id: 6822b5441f775dd12f7534a7
paper_authors.name: Zhenru Lin
paper_authors._id: 6822b5441f775dd12f7534a8
paper_authors.name: Zhichao Song
paper_authors._id: 6822b5441f775dd12f7534a9
paper_authors.name: Zihao Yue
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-13T08:11:53.954Z
paper_authors._id: 6517f0df593b3af3120d242e
paper_authors.avatarUrl: /avatars/bc28ceb48b206502d8cf9e1c2e130066.svg
paper_authors.fullname: Zihao Yue
paper_authors.type: user
paper_authors.user: yuezih
paper_title: MiMo: Unlocking the Reasoning Potential of Language Model -- From
    Pretraining to Posttraining
paper_summary: We present MiMo-7B, a large language model born for reasoning tasks, with
  optimization across both pre-training and post-training stages. During
  pre-training, we enhance the data preprocessing pipeline and employ a
  three-stage data mixing strategy to strengthen the base model's reasoning
  potential. MiMo-7B-Base is pre-trained on 25 trillion tokens, with additional
  Multi-Token Prediction objective for enhanced performance and accelerated
  inference speed. During post-training, we curate a dataset of 130K verifiable
  mathematics and programming problems for reinforcement learning, integrating a
  test-difficulty-driven code-reward scheme to alleviate sparse-reward issues and
  employing strategic data resampling to stabilize training. Extensive
  evaluations show that MiMo-7B-Base possesses exceptional reasoning potential,
  outperforming even much larger 32B models. The final RL-tuned model,
  MiMo-7B-RL, achieves superior performance on mathematics, code and general
  reasoning tasks, surpassing the performance of OpenAI o1-mini. The model
  checkpoints are available at https://github.com/xiaomimimo/MiMo.
paper_ai_summary: MiMo-7B, a large language model optimized for reasoning tasks, enhances pre-training with data mixing and Multi-Token Prediction, and post-training with reinforcement learning on math and programming problems, achieving superior performance over larger models.
paper_ai_keywords: large language model pre-training data preprocessing three-stage data mixing strategy Multi-Token Prediction post-training reinforcement learning test-difficulty-driven code-reward scheme strategic data resampling extensive evaluations mathematics programming problems general reasoning tasks