title: ChatGLM-Math: Improving Math Problem-Solving in Large Language Models
    with a Self-Critique Pipeline
summary: Large language models (LLMs) have shown excellent mastering of human
  language, but still struggle in real-world applications that require
  mathematical problem-solving. While many strategies and datasets to enhance
  LLMs' mathematics are developed, it remains a challenge to simultaneously
  maintain and improve both language and mathematical capabilities in deployed
  LLM systems.In this work, we tailor the Self-Critique pipeline, which addresses
  the challenge in the feedback learning stage of LLM alignment. We first train a
  general Math-Critique model from the LLM itself to provide feedback signals.
  Then, we sequentially employ rejective fine-tuning and direct preference
  optimization over the LLM's own generations for data collection. Based on
  ChatGLM3-32B, we conduct a series of experiments on both academic and our newly
  created challenging dataset, MathUserEval. Results show that our pipeline
  significantly enhances the LLM's mathematical problem-solving while still
  improving its language ability, outperforming LLMs that could be two times
  larger. Related techniques have been deployed to
  ChatGLM\url{https://chatglm.cn}, an online serving LLM. Related
  evaluation dataset and scripts are released at
  https://github.com/THUDM/ChatGLM-Math.
paper_authors._id: 660e1b8583bce1a92c00b84d
paper_authors.name: Yifan Xu
paper_authors._id: 660e1b8583bce1a92c00b84e
paper_authors.name: Xiao Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-05T16:12:37.242Z
paper_authors._id: 62f098602ca4d32a7cd87aba
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1659934844914-62f098602ca4d32a7cd87aba.jpeg
paper_authors.fullname: Xiao Liu
paper_authors.type: user
paper_authors.user: ShawLiu
paper_authors._id: 660e1b8583bce1a92c00b84f
paper_authors.name: Xinghan Liu
paper_authors._id: 660e1b8583bce1a92c00b850
paper_authors.name: Zhenyu Hou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-23T09:22:19.364Z
paper_authors._id: 62b196646d3d059f40c3df19
paper_authors.avatarUrl: /avatars/dbddf54ae949437223f3a438d30ef653.svg
paper_authors.fullname: Zhenyu Hou
paper_authors.type: user
paper_authors.user: think2try
paper_authors._id: 660e1b8583bce1a92c00b851
paper_authors.name: Yueyan Li
paper_authors._id: 660e1b8583bce1a92c00b852
paper_authors.name: Xiaohan Zhang
paper_authors._id: 660e1b8583bce1a92c00b853
paper_authors.name: Zihan Wang
paper_authors._id: 660e1b8583bce1a92c00b854
paper_authors.name: Aohan Zeng
paper_authors._id: 660e1b8583bce1a92c00b855
paper_authors.name: Zhengxiao Du
paper_authors._id: 660e1b8583bce1a92c00b856
paper_authors.name: Wenyi Zhao
paper_authors._id: 660e1b8583bce1a92c00b857
paper_authors.name: Jie Tang
paper_authors._id: 660e1b8583bce1a92c00b858
paper_authors.name: Yuxiao Dong
paper_title: ChatGLM-Math: Improving Math Problem-Solving in Large Language Models
    with a Self-Critique Pipeline
paper_summary: Large language models (LLMs) have shown excellent mastering of human
  language, but still struggle in real-world applications that require
  mathematical problem-solving. While many strategies and datasets to enhance
  LLMs' mathematics are developed, it remains a challenge to simultaneously
  maintain and improve both language and mathematical capabilities in deployed
  LLM systems.In this work, we tailor the Self-Critique pipeline, which addresses
  the challenge in the feedback learning stage of LLM alignment. We first train a
  general Math-Critique model from the LLM itself to provide feedback signals.
  Then, we sequentially employ rejective fine-tuning and direct preference
  optimization over the LLM's own generations for data collection. Based on
  ChatGLM3-32B, we conduct a series of experiments on both academic and our newly
  created challenging dataset, MathUserEval. Results show that our pipeline
  significantly enhances the LLM's mathematical problem-solving while still
  improving its language ability, outperforming LLMs that could be two times
  larger. Related techniques have been deployed to
  ChatGLM\url{https://chatglm.cn}, an online serving LLM. Related
  evaluation dataset and scripts are released at
  https://github.com/THUDM/ChatGLM-Math.
paper_ai_summary: A self-critique pipeline enhances LLMs' mathematical problem-solving and language abilities through Math-Critique, rejective fine-tuning, and direct preference optimization, outperforming larger models.
paper_ai_keywords: Self-Critique pipeline Math-Critique model rejective fine-tuning direct preference optimization ChatGLM3-32B MathUserEval LLM alignment