title: MathFusion: Enhancing Mathematic Problem-solving of LLM through
    Instruction Fusion
summary: Large Language Models (LLMs) have shown impressive progress in mathematical
  reasoning. While data augmentation is promising to enhance mathematical
  problem-solving ability, current approaches are predominantly limited to
  instance-level modifications-such as rephrasing or generating syntactic
  variations-which fail to capture and leverage the intrinsic relational
  structures inherent in mathematical knowledge. Inspired by human learning
  processes, where mathematical proficiency develops through systematic exposure
  to interconnected concepts, we introduce MathFusion, a novel framework that
  enhances mathematical reasoning through cross-problem instruction synthesis.
  MathFusion implements this through three fusion strategies: (1) sequential
  fusion, which chains related problems to model solution dependencies; (2)
  parallel fusion, which combines analogous problems to reinforce conceptual
  understanding; and (3) conditional fusion, which creates context-aware
  selective problems to enhance reasoning flexibility. By applying these
  strategies, we generate a new dataset, MathFusionQA, followed by
  fine-tuning models (DeepSeekMath-7B, Mistral-7B, Llama3-8B) on it. Experimental
  results demonstrate that MathFusion achieves substantial improvements in
  mathematical reasoning while maintaining high data efficiency, boosting
  performance by 18.0 points in accuracy across diverse benchmarks while
  requiring only 45K additional synthetic instructions, representing a
  substantial improvement over traditional single-instruction approaches. Our
  datasets, models, and code are publicly available at
  https://github.com/QizhiPei/mathfusion.
paper_authors._id: 67dcd33626989570158ce8cf
paper_authors.name: Qizhi Pei
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-21T13:00:02.682Z
paper_authors._id: 6397f6081323f19c578f142e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6397f6081323f19c578f142e/it7FYYKjlLX8wSsMLm8EO.jpeg
paper_authors.fullname: QizhiPei
paper_authors.type: user
paper_authors.user: QizhiPei
paper_authors._id: 67dcd33626989570158ce8d0
paper_authors.name: Lijun Wu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-21T13:00:09.197Z
paper_authors._id: 643e60d96db6ba8c5ee177ad
paper_authors.avatarUrl: /avatars/73ac7740e462ba0b53a2f2480d9f1e3e.svg
paper_authors.fullname: Lijun Wu
paper_authors.type: user
paper_authors.user: apeters
paper_authors._id: 67dcd33626989570158ce8d1
paper_authors.name: Zhuoshi Pan
paper_authors._id: 67dcd33626989570158ce8d2
paper_authors.name: Yu Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-29T08:00:00.327Z
paper_authors._id: 671b852aa4fa4f8f5fb5404c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/671b852aa4fa4f8f5fb5404c/TDLsgP8WgKW-qaA8Ys-iJ.jpeg
paper_authors.fullname: YU LI
paper_authors.type: user
paper_authors.user: yu0226
paper_authors._id: 67dcd33626989570158ce8d3
paper_authors.name: Honglin Lin
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-21T13:00:16.585Z
paper_authors._id: 640d99628512ec51d7ef71c7
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/640d99628512ec51d7ef71c7/fcBkqnxfxuuuZTqfN_BGy.jpeg
paper_authors.fullname: Honglin Lin
paper_authors.type: user
paper_authors.user: LHL3341
paper_authors._id: 67dcd33626989570158ce8d4
paper_authors.name: Chenlin Ming
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-21T13:00:29.586Z
paper_authors._id: 677e133ee86d0754dc7ce296
paper_authors.avatarUrl: /avatars/c16511c1876b50c2d049925c5f320d15.svg
paper_authors.fullname: mingchenlin
paper_authors.type: user
paper_authors.user: mingchenlin2025
paper_authors._id: 67dcd33626989570158ce8d5
paper_authors.name: Xin Gao
paper_authors._id: 67dcd33626989570158ce8d6
paper_authors.name: Conghui He
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-21T13:00:36.973Z
paper_authors._id: 63f9fca8d4349b157a109eec
paper_authors.avatarUrl: /avatars/fa1f2ae7972d7cde99dab178136ccbb0.svg
paper_authors.fullname: Conghui He
paper_authors.type: user
paper_authors.user: conghui
paper_authors._id: 67dcd33626989570158ce8d7
paper_authors.name: Rui Yan
paper_title: MathFusion: Enhancing Mathematic Problem-solving of LLM through
    Instruction Fusion
paper_summary: Large Language Models (LLMs) have shown impressive progress in mathematical
  reasoning. While data augmentation is promising to enhance mathematical
  problem-solving ability, current approaches are predominantly limited to
  instance-level modifications-such as rephrasing or generating syntactic
  variations-which fail to capture and leverage the intrinsic relational
  structures inherent in mathematical knowledge. Inspired by human learning
  processes, where mathematical proficiency develops through systematic exposure
  to interconnected concepts, we introduce MathFusion, a novel framework that
  enhances mathematical reasoning through cross-problem instruction synthesis.
  MathFusion implements this through three fusion strategies: (1) sequential
  fusion, which chains related problems to model solution dependencies; (2)
  parallel fusion, which combines analogous problems to reinforce conceptual
  understanding; and (3) conditional fusion, which creates context-aware
  selective problems to enhance reasoning flexibility. By applying these
  strategies, we generate a new dataset, MathFusionQA, followed by
  fine-tuning models (DeepSeekMath-7B, Mistral-7B, Llama3-8B) on it. Experimental
  results demonstrate that MathFusion achieves substantial improvements in
  mathematical reasoning while maintaining high data efficiency, boosting
  performance by 18.0 points in accuracy across diverse benchmarks while
  requiring only 45K additional synthetic instructions, representing a
  substantial improvement over traditional single-instruction approaches. Our
  datasets, models, and code are publicly available at
  https://github.com/QizhiPei/mathfusion.
paper_ai_summary: A new framework, MathFusion, enhances mathematical reasoning by synthesizing cross-problem instructions, leading to significant improvements in accuracy with fewer additional synthetic instructions.
paper_ai_keywords: Large Language Models data augmentation mathematical reasoning instance-level modifications intrinsic relational structures systematic exposure interconnected concepts cross-problem instruction synthesis sequential fusion parallel fusion conditional fusion MathFusionQA fine-tuning DeepSeekMath-7B Mistral-7B Llama3-8B data efficiency