ChatGLM-Math: Improving Math Problem-Solving in Large Language Models
  with a Self-Critique Pipeline
Large language models (LLMs) have shown excellent mastering of human
language, but still struggle in real-world applications that require
mathematical problem-solving. While many strategies and datasets to enhance
LLMs' mathematics are developed, it remains a challenge to simultaneously
maintain and improve both language and mathematical capabilities in deployed
LLM systems.In this work, we tailor the Self-Critique pipeline, which addresses
the challenge in the feedback learning stage of LLM alignment. We first train a
general Math-Critique model from the LLM itself to provide feedback signals.
Then, we sequentially employ rejective fine-tuning and direct preference
optimization over the LLM's own generations for data collection. Based on
ChatGLM3-32B, we conduct a series of experiments on both academic and our newly
created challenging dataset, MathUserEval. Results show that our pipeline
significantly enhances the LLM's mathematical problem-solving while still
improving its language ability, outperforming LLMs that could be two times
larger. Related techniques have been deployed to
ChatGLM\url{https://chatglm.cn}, an online serving LLM. Related
evaluation dataset and scripts are released at
https://github.com/THUDM/ChatGLM-Math.
660e1b8583bce1a92c00b84d
Yifan Xu
660e1b8583bce1a92c00b84e
Xiao Liu
claimed_verified
2024-11-05T16:12:37.242Z
62f098602ca4d32a7cd87aba
https://cdn-avatars.huggingface.co/v1/production/uploads/1659934844914-62f098602ca4d32a7cd87aba.jpeg
Xiao Liu
user
ShawLiu
660e1b8583bce1a92c00b84f
Xinghan Liu
660e1b8583bce1a92c00b850
Zhenyu Hou
claimed_verified
2025-01-23T09:22:19.364Z
62b196646d3d059f40c3df19
/avatars/dbddf54ae949437223f3a438d30ef653.svg
Zhenyu Hou
user
think2try
660e1b8583bce1a92c00b851
Yueyan Li
660e1b8583bce1a92c00b852
Xiaohan Zhang
660e1b8583bce1a92c00b853
Zihan Wang
660e1b8583bce1a92c00b854
Aohan Zeng
660e1b8583bce1a92c00b855
Zhengxiao Du
660e1b8583bce1a92c00b856
Wenyi Zhao
660e1b8583bce1a92c00b857
Jie Tang
660e1b8583bce1a92c00b858
Yuxiao Dong
ChatGLM-Math: Improving Math Problem-Solving in Large Language Models
  with a Self-Critique Pipeline
Large language models (LLMs) have shown excellent mastering of human
language, but still struggle in real-world applications that require
mathematical problem-solving. While many strategies and datasets to enhance
LLMs' mathematics are developed, it remains a challenge to simultaneously
maintain and improve both language and mathematical capabilities in deployed
LLM systems.In this work, we tailor the Self-Critique pipeline, which addresses
the challenge in the feedback learning stage of LLM alignment. We first train a
general Math-Critique model from the LLM itself to provide feedback signals.
Then, we sequentially employ rejective fine-tuning and direct preference
optimization over the LLM's own generations for data collection. Based on
ChatGLM3-32B, we conduct a series of experiments on both academic and our newly
created challenging dataset, MathUserEval. Results show that our pipeline
significantly enhances the LLM's mathematical problem-solving while still
improving its language ability, outperforming LLMs that could be two times
larger. Related techniques have been deployed to
ChatGLM\url{https://chatglm.cn}, an online serving LLM. Related
evaluation dataset and scripts are released at
https://github.com/THUDM/ChatGLM-Math.
A self-critique pipeline enhances LLMs' mathematical problem-solving and language abilities through Math-Critique, rejective fine-tuning, and direct preference optimization, outperforming larger models.
Self-Critique pipeline Math-Critique model rejective fine-tuning direct preference optimization ChatGLM3-32B MathUserEval LLM alignment