title: Unlocking Efficient Long-to-Short LLM Reasoning with Model Merging
summary: The transition from System 1 to System 2 reasoning in large language models
  (LLMs) has marked significant advancements in handling complex tasks through
  deliberate, iterative thinking. However, this progress often comes at the cost
  of efficiency, as models tend to overthink, generating redundant reasoning
  steps without proportional improvements in output quality. Long-to-Short (L2S)
  reasoning has emerged as a promising solution to this challenge, aiming to
  balance reasoning depth with practical efficiency. While existing approaches,
  such as supervised fine-tuning (SFT), reinforcement learning (RL), and prompt
  engineering, have shown potential, they are either computationally expensive or
  unstable. Model merging, on the other hand, offers a cost-effective and robust
  alternative by integrating the quick-thinking capabilities of System 1 models
  with the methodical reasoning of System 2 models. In this work, we present a
  comprehensive empirical study on model merging for L2S reasoning, exploring
  diverse methodologies, including task-vector-based, SVD-based, and
  activation-informed merging. Our experiments reveal that model merging can
  reduce average response length by up to 55% while preserving or even improving
  baseline performance. We also identify a strong correlation between model scale
  and merging efficacy with extensive evaluations on 1.5B/7B/14B/32B models.
  Furthermore, we investigate the merged model's ability to self-critique and
  self-correct, as well as its adaptive response length based on task complexity.
  Our findings highlight model merging as a highly efficient and effective
  paradigm for L2S reasoning, offering a practical solution to the overthinking
  problem while maintaining the robustness of System 2 reasoning. This work can
  be found on Github https://github.com/hahahawu/Long-to-Short-via-Model-Merging.
paper_authors._id: 67e5608e345a214efb382ad5
paper_authors.name: Han Wu
paper_authors._id: 67e5608e345a214efb382ad6
paper_authors.name: Yuxuan Yao
paper_authors._id: 67e5608e345a214efb382ad7
paper_authors.name: Shuqi Liu
paper_authors._id: 67e5608e345a214efb382ad8
paper_authors.name: Zehua Liu
paper_authors._id: 67e5608e345a214efb382ad9
paper_authors.name: Xiaojin Fu
paper_authors._id: 67e5608e345a214efb382ada
paper_authors.name: Xiongwei Han
paper_authors._id: 67e5608e345a214efb382adb
paper_authors.name: Xing Li
paper_authors._id: 67e5608e345a214efb382adc
paper_authors.name: Hui-Ling Zhen
paper_authors._id: 67e5608e345a214efb382add
paper_authors.name: Tao Zhong
paper_authors._id: 67e5608e345a214efb382ade
paper_authors.name: Mingxuan Yuan
paper_title: Unlocking Efficient Long-to-Short LLM Reasoning with Model Merging
paper_summary: The transition from System 1 to System 2 reasoning in large language models
  (LLMs) has marked significant advancements in handling complex tasks through
  deliberate, iterative thinking. However, this progress often comes at the cost
  of efficiency, as models tend to overthink, generating redundant reasoning
  steps without proportional improvements in output quality. Long-to-Short (L2S)
  reasoning has emerged as a promising solution to this challenge, aiming to
  balance reasoning depth with practical efficiency. While existing approaches,
  such as supervised fine-tuning (SFT), reinforcement learning (RL), and prompt
  engineering, have shown potential, they are either computationally expensive or
  unstable. Model merging, on the other hand, offers a cost-effective and robust
  alternative by integrating the quick-thinking capabilities of System 1 models
  with the methodical reasoning of System 2 models. In this work, we present a
  comprehensive empirical study on model merging for L2S reasoning, exploring
  diverse methodologies, including task-vector-based, SVD-based, and
  activation-informed merging. Our experiments reveal that model merging can
  reduce average response length by up to 55% while preserving or even improving
  baseline performance. We also identify a strong correlation between model scale
  and merging efficacy with extensive evaluations on 1.5B/7B/14B/32B models.
  Furthermore, we investigate the merged model's ability to self-critique and
  self-correct, as well as its adaptive response length based on task complexity.
  Our findings highlight model merging as a highly efficient and effective
  paradigm for L2S reasoning, offering a practical solution to the overthinking
  problem while maintaining the robustness of System 2 reasoning. This work can
  be found on Github https://github.com/hahahawu/Long-to-Short-via-Model-Merging.
paper_ai_summary: Model merging integrates quick and methodical reasoning to enhance efficiency and maintain performance in large language models, reducing redundant steps without losing output quality.
paper_ai_keywords: System 1 System 2 L2S reasoning supervised fine-tuning reinforcement learning prompt engineering task-vector-based merging SVD-based merging activation-informed merging self-critique self-correction