title: Mind the Gap: Bridging Thought Leap for Improved Chain-of-Thought Tuning
summary: Large language models (LLMs) have achieved remarkable progress on
  mathematical tasks through Chain-of-Thought (CoT) reasoning. However, existing
  mathematical CoT datasets often suffer from Thought Leaps due to experts
  omitting intermediate steps, which negatively impacts model learning and
  generalization. We propose the CoT Thought Leap Bridge Task, which aims to
  automatically detect leaps and generate missing intermediate reasoning steps to
  restore the completeness and coherence of CoT. To facilitate this, we
  constructed a specialized training dataset called ScaleQM+, based on the
  structured ScaleQuestMath dataset, and trained CoT-Bridge to bridge thought
  leaps. Through comprehensive experiments on mathematical reasoning benchmarks,
  we demonstrate that models fine-tuned on bridged datasets consistently
  outperform those trained on original datasets, with improvements of up to
  +5.87% on NuminaMath. Our approach effectively enhances distilled data (+3.02%)
  and provides better starting points for reinforcement learning (+3.1%),
  functioning as a plug-and-play module compatible with existing optimization
  techniques. Furthermore, CoT-Bridge demonstrate improved generalization to
  out-of-domain logical reasoning tasks, confirming that enhancing reasoning
  completeness yields broadly applicable benefits.
paper_authors._id: 682f474c9ee0bb0cc953b885
paper_authors.name: Haolei Xu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-26T08:17:23.733Z
paper_authors._id: 6692aff88db712bad780f02a
paper_authors.avatarUrl: /avatars/5dc4b1c27c70f6a64864711dbff4910f.svg
paper_authors.fullname: xhl
paper_authors.type: user
paper_authors.user: zjuxhl
paper_authors._id: 682f474c9ee0bb0cc953b886
paper_authors.name: Yuchen Yan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-26T08:17:21.363Z
paper_authors._id: 64098738342c26884c792c93
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64098738342c26884c792c93/SxBUd-wLrl-PjQsrVYJte.jpeg
paper_authors.fullname: Yuchen Yan
paper_authors.type: user
paper_authors.user: yanyc
paper_authors._id: 682f474c9ee0bb0cc953b887
paper_authors.name: Yongliang Shen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-26T08:17:18.955Z
paper_authors._id: 5e1058e9fcf41d740b69966d
paper_authors.avatarUrl: /avatars/ce74839ba871f2b54313a670a233ba82.svg
paper_authors.fullname: Yongliang Shen
paper_authors.type: user
paper_authors.user: tricktreat
paper_authors._id: 682f474c9ee0bb0cc953b888
paper_authors.name: Wenqi Zhang
paper_authors._id: 682f474c9ee0bb0cc953b889
paper_authors.name: Guiyang Hou
paper_authors._id: 682f474c9ee0bb0cc953b88a
paper_authors.name: Shengpei Jiang
paper_authors._id: 682f474c9ee0bb0cc953b88b
paper_authors.name: Kaitao Song
paper_authors._id: 682f474c9ee0bb0cc953b88c
paper_authors.name: Weiming Lu
paper_authors._id: 682f474c9ee0bb0cc953b88d
paper_authors.name: Jun Xiao
paper_authors._id: 682f474c9ee0bb0cc953b88e
paper_authors.name: Yueting Zhuang
paper_title: Mind the Gap: Bridging Thought Leap for Improved Chain-of-Thought Tuning
paper_summary: Large language models (LLMs) have achieved remarkable progress on
  mathematical tasks through Chain-of-Thought (CoT) reasoning. However, existing
  mathematical CoT datasets often suffer from Thought Leaps due to experts
  omitting intermediate steps, which negatively impacts model learning and
  generalization. We propose the CoT Thought Leap Bridge Task, which aims to
  automatically detect leaps and generate missing intermediate reasoning steps to
  restore the completeness and coherence of CoT. To facilitate this, we
  constructed a specialized training dataset called ScaleQM+, based on the
  structured ScaleQuestMath dataset, and trained CoT-Bridge to bridge thought
  leaps. Through comprehensive experiments on mathematical reasoning benchmarks,
  we demonstrate that models fine-tuned on bridged datasets consistently
  outperform those trained on original datasets, with improvements of up to
  +5.87% on NuminaMath. Our approach effectively enhances distilled data (+3.02%)
  and provides better starting points for reinforcement learning (+3.1%),
  functioning as a plug-and-play module compatible with existing optimization
  techniques. Furthermore, CoT-Bridge demonstrate improved generalization to
  out-of-domain logical reasoning tasks, confirming that enhancing reasoning
  completeness yields broadly applicable benefits.
paper_ai_summary: A model for detecting and generating missing intermediate steps in mathematical Chain-of-Thought reasoning improves performance and generalization on mathematical and logical reasoning tasks.
paper_ai_keywords: Chain-of-Thought reasoning thought leaps ScaleQM+ ScaleQuestMath NuminaMath distilled data reinforcement learning generalization logical reasoning tasks