MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical
  Reasoning
The recently released GPT-4 Code Interpreter has demonstrated remarkable
proficiency in solving challenging math problems, primarily attributed to its
ability to seamlessly reason with natural language, generate code, execute
code, and continue reasoning based on the execution output. In this paper, we
present a method to fine-tune open-source language models, enabling them to use
code for modeling and deriving math equations and, consequently, enhancing
their mathematical reasoning abilities. We propose a method of generating novel
and high-quality datasets with math problems and their code-based solutions,
referred to as MathCodeInstruct. Each solution interleaves natural language,
code, and execution results. We also introduce a customized supervised
fine-tuning and inference approach. This approach yields the MathCoder models,
a family of models capable of generating code-based solutions for solving
challenging math problems. Impressively, the MathCoder models achieve
state-of-the-art scores among open-source LLMs on the MATH (45.2%) and GSM8K
(83.9%) datasets, substantially outperforming other open-source alternatives.
Notably, the MathCoder model not only surpasses ChatGPT-3.5 and PaLM-2 on GSM8K
and MATH but also outperforms GPT-4 on the competition-level MATH dataset. The
dataset and models will be released at https://github.com/mathllm/MathCoder.
651f6771be34a5f2cf5e0990
Ke Wang
claimed_verified
2024-10-11T07:47:36.483Z
64d592c28767727dffa1f002
/avatars/fe38bcac944a2742dc12c624e62d24ef.svg
WangKe
user
scikkk
651f6771be34a5f2cf5e0991
Houxing Ren
admin_assigned
2023-10-09T08:26:55.129Z
63e4b63dd6278c161be47f4b
https://cdn-avatars.huggingface.co/v1/production/uploads/63e4b63dd6278c161be47f4b/u0y1_o9WBqPJh8C8dimJ8.jpeg
Houxing Ren
user
Houxing
651f6771be34a5f2cf5e0992
Aojun Zhou
651f6771be34a5f2cf5e0993
Zimu Lu
admin_assigned
2023-10-09T08:27:12.432Z
64b0bfef2f2f9c345b87e673
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/7O3zCr_IyGvtz0BUdKNKi.jpeg
Zimu Lu
user
luzimu
651f6771be34a5f2cf5e0994
Sichun Luo
651f6771be34a5f2cf5e0995
Weikang Shi
admin_assigned
2023-10-09T08:27:29.478Z
6337e1d1107c4835a04c8607
/avatars/e85353b34c3da6efef84951934811fee.svg
Weikang Shi
user
shiwk20
651f6771be34a5f2cf5e0996
Renrui Zhang
claimed_verified
2024-03-22T08:40:18.584Z
645b8b2687c79b6ec0bb3b7a
/avatars/00a9db32a42dc950112bf2593bb109cb.svg
Renrui
user
ZrrSkywalker
651f6771be34a5f2cf5e0997
Linqi Song
651f6771be34a5f2cf5e0998
Mingjie Zhan
651f6771be34a5f2cf5e0999
Hongsheng Li
admin_assigned
2023-10-09T08:28:15.657Z
64d97a3a67c967b01512dd1c
/avatars/8a885f9290d4f42de4c481bcfcae8d85.svg
Hongsheng Liu
user
hsliu
MathCoder: Seamless Code Integration in LLMs for Enhanced Mathematical
  Reasoning
The recently released GPT-4 Code Interpreter has demonstrated remarkable
proficiency in solving challenging math problems, primarily attributed to its
ability to seamlessly reason with natural language, generate code, execute
code, and continue reasoning based on the execution output. In this paper, we
present a method to fine-tune open-source language models, enabling them to use
code for modeling and deriving math equations and, consequently, enhancing
their mathematical reasoning abilities. We propose a method of generating novel
and high-quality datasets with math problems and their code-based solutions,
referred to as MathCodeInstruct. Each solution interleaves natural language,
code, and execution results. We also introduce a customized supervised
fine-tuning and inference approach. This approach yields the MathCoder models,
a family of models capable of generating code-based solutions for solving
challenging math problems. Impressively, the MathCoder models achieve
state-of-the-art scores among open-source LLMs on the MATH (45.2%) and GSM8K
(83.9%) datasets, substantially outperforming other open-source alternatives.
Notably, the MathCoder model not only surpasses ChatGPT-3.5 and PaLM-2 on GSM8K
and MATH but also outperforms GPT-4 on the competition-level MATH dataset. The
dataset and models will be released at https://github.com/mathllm/MathCoder.
A method is presented for fine-tuning language models to solve math problems using code, achieving state-of-the-art results on MATH and GSM8K datasets.
fine-tuning language models MathCodeInstruct code-based solutions supervised fine-tuning inference MathCoder MATH dataset GSM8K dataset ChatGPT-3.5 PaLM-2 GPT-4