title: InternLM-Math: Open Math Large Language Models Toward Verifiable
    Reasoning
summary: The math abilities of large language models can represent their abstract
  reasoning ability. In this paper, we introduce and open-source our math
  reasoning LLMs InternLM-Math which is continue pre-trained from InternLM2. We
  unify chain-of-thought reasoning, reward modeling, formal reasoning, data
  augmentation, and code interpreter in a unified seq2seq format and supervise
  our model to be a versatile math reasoner, verifier, prover, and augmenter.
  These abilities can be used to develop the next math LLMs or self-iteration.
  InternLM-Math obtains open-sourced state-of-the-art performance under the
  setting of in-context learning, supervised fine-tuning, and code-assisted
  reasoning in various informal and formal benchmarks including GSM8K, MATH,
  Hungary math exam, MathBench-ZH, and MiniF2F. Our pre-trained model achieves
  30.3 on the MiniF2F test set without fine-tuning. We further explore how to use
  LEAN to solve math problems and study its performance under the setting of
  multi-task learning which shows the possibility of using LEAN as a unified
  platform for solving and proving in math. Our models, codes, and data are
  released at https://github.com/InternLM/InternLM-Math.
paper_authors._id: 65c980181bc880f366623a73
paper_authors.name: Huaiyuan Ying
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-12T16:41:06.588Z
paper_authors._id: 63078192a642348dc5e3a77c
paper_authors.avatarUrl: /avatars/367832fd370b6720cdd7c9c6c5af9d21.svg
paper_authors.fullname: Huaiyuan Ying
paper_authors.type: user
paper_authors.user: yinghy18
paper_authors._id: 65c980181bc880f366623a74
paper_authors.name: Shuo Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-12T16:41:37.472Z
paper_authors._id: 6334f2f1259c518276efa730
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6334f2f1259c518276efa730/z_SH_OBkDyj4RCN9mqsKS.jpeg
paper_authors.fullname: Shuo Zhang
paper_authors.type: user
paper_authors.user: Meteonis
paper_authors._id: 65c980181bc880f366623a75
paper_authors.name: Linyang Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-18T09:09:40.970Z
paper_authors._id: 6698ad87a9778d20ff373142
paper_authors.avatarUrl: /avatars/5e3546d124e8c4f4636a41eab6f0cf68.svg
paper_authors.fullname: linyangli
paper_authors.type: user
paper_authors.user: linyangli
paper_authors._id: 65c980181bc880f366623a76
paper_authors.name: Zhejian Zhou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-14T07:49:16.573Z
paper_authors._id: 61f650d9bf54777a520c599d
paper_authors.avatarUrl: /avatars/9e7b4fc709835c0934bf2671298c9b33.svg
paper_authors.fullname: Jeremiah Zhou
paper_authors.type: user
paper_authors.user: JeremiahZ
paper_authors._id: 65c980181bc880f366623a77
paper_authors.name: Yunfan Shao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-12T16:42:10.240Z
paper_authors._id: 613b08710fcccdd3b430478a
paper_authors.avatarUrl: /avatars/6bb368110cd50996b26046cd09ab9bcc.svg
paper_authors.fullname: Yunfan Shao
paper_authors.type: user
paper_authors.user: yf
paper_authors._id: 65c980181bc880f366623a78
paper_authors.name: Zhaoye Fei
paper_authors._id: 65c980181bc880f366623a79
paper_authors.name: Yichuan Ma
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-12T16:42:26.178Z
paper_authors._id: 6502c96abf479c8327323fd1
paper_authors.avatarUrl: /avatars/a5b93638cba659948c4cfc00790e3a1e.svg
paper_authors.fullname: 马逸川
paper_authors.type: user
paper_authors.user: YichuanMa
paper_authors._id: 65c980181bc880f366623a7a
paper_authors.name: Jiawei Hong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-12T16:42:33.590Z
paper_authors._id: 65b14bad54fa188a2f8134a4
paper_authors.avatarUrl: /avatars/b54b1152f94ed25b9180d64cbc686f75.svg
paper_authors.fullname: Jia Wei Hong
paper_authors.type: user
paper_authors.user: jiaweihong
paper_authors._id: 65c980181bc880f366623a7b
paper_authors.name: Kuikun Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-08-02T07:35:21.969Z
paper_authors._id: 63fd691794cc8f815d50c112
paper_authors.avatarUrl: /avatars/87305d1cbfcc717e910ccdfaf0568f80.svg
paper_authors.fullname: liu
paper_authors.type: user
paper_authors.user: Harold-lkk
paper_authors._id: 65c980181bc880f366623a7c
paper_authors.name: Ziyi Wang
paper_authors._id: 65c980181bc880f366623a7d
paper_authors.name: Yudong Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-09T07:24:10.553Z
paper_authors._id: 63be286fb3b8c44f8cecc16f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63be286fb3b8c44f8cecc16f/1CIkfEKoTnBYdYDSuQ8AT.jpeg
paper_authors.fullname: Yudong Wang
paper_authors.type: user
paper_authors.user: BigDong
paper_authors._id: 65c980181bc880f366623a7e
paper_authors.name: Zijian Wu
paper_authors._id: 65c980181bc880f366623a7f
paper_authors.name: Shuaibin Li
paper_authors._id: 65c980181bc880f366623a80
paper_authors.name: Fengzhe Zhou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-12T16:45:35.972Z
paper_authors._id: 63f04fb94a788ed1dd89daf4
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63f04fb94a788ed1dd89daf4/5XNPePED6fTMH0SAM2lhj.jpeg
paper_authors.fullname: zhoufengzhe
paper_authors.type: user
paper_authors.user: Leymore
paper_authors._id: 65c980181bc880f366623a81
paper_authors.name: Hongwei Liu
paper_authors._id: 65c980181bc880f366623a82
paper_authors.name: Songyang Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-12T16:46:33.141Z
paper_authors._id: 630716d11801ecc7d2595021
paper_authors.avatarUrl: /avatars/2d36a880ce4a3cf7efc5ff3987dbeaf3.svg
paper_authors.fullname: Songyang Zhang
paper_authors.type: user
paper_authors.user: zsytony
paper_authors._id: 65c980181bc880f366623a83
paper_authors.name: Wenwei Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-12T16:46:39.491Z
paper_authors._id: 64e8505321540e1da3226b54
paper_authors.avatarUrl: /avatars/18958b8406d1ce492b54c1c839f18c54.svg
paper_authors.fullname: Wenwei Zhang
paper_authors.type: user
paper_authors.user: ZwwWayne
paper_authors._id: 65c980181bc880f366623a84
paper_authors.name: Hang Yan
paper_authors._id: 65c980181bc880f366623a85
paper_authors.name: Xipeng Qiu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-12T16:47:13.100Z
paper_authors._id: 61457b8deff2c9fdb4de4988
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1632381702899-61457b8deff2c9fdb4de4988.jpeg
paper_authors.fullname: Xipeng Qiu
paper_authors.type: user
paper_authors.user: xpqiu
paper_authors._id: 65c980181bc880f366623a86
paper_authors.name: Jiayu Wang
paper_authors._id: 65c980181bc880f366623a87
paper_authors.name: Kai Chen
paper_authors._id: 65c980181bc880f366623a88
paper_authors.name: Dahua Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-12T16:48:36.682Z
paper_authors._id: 636317ed80c1a705a6eff396
paper_authors.avatarUrl: /avatars/3db090e101b916d9256d0d3e043db71d.svg
paper_authors.fullname: Dahua Lin
paper_authors.type: user
paper_authors.user: lindahua
paper_title: InternLM-Math: Open Math Large Language Models Toward Verifiable
    Reasoning
paper_summary: The math abilities of large language models can represent their abstract
  reasoning ability. In this paper, we introduce and open-source our math
  reasoning LLMs InternLM-Math which is continue pre-trained from InternLM2. We
  unify chain-of-thought reasoning, reward modeling, formal reasoning, data
  augmentation, and code interpreter in a unified seq2seq format and supervise
  our model to be a versatile math reasoner, verifier, prover, and augmenter.
  These abilities can be used to develop the next math LLMs or self-iteration.
  InternLM-Math obtains open-sourced state-of-the-art performance under the
  setting of in-context learning, supervised fine-tuning, and code-assisted
  reasoning in various informal and formal benchmarks including GSM8K, MATH,
  Hungary math exam, MathBench-ZH, and MiniF2F. Our pre-trained model achieves
  30.3 on the MiniF2F test set without fine-tuning. We further explore how to use
  LEAN to solve math problems and study its performance under the setting of
  multi-task learning which shows the possibility of using LEAN as a unified
  platform for solving and proving in math. Our models, codes, and data are
  released at https://github.com/InternLM/InternLM-Math.
paper_ai_summary: InternLM-Math, a math reasoning LLM, achieves state-of-the-art performance in various benchmarks by integrating chain-of-thought reasoning, reward modeling, formal reasoning, data augmentation, and code interpretation.
paper_ai_keywords: chain-of-thought reasoning reward modeling formal reasoning data augmentation code interpreter seq2seq math reasoner verifier prover augmenter in-context learning supervised fine-tuning code-assisted reasoning GSM8K MATH MiniF2F LEAN multi-task learning