title: O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning
summary: Recently, long-thought reasoning LLMs, such as OpenAI's O1, adopt extended
  reasoning processes similar to how humans ponder over complex problems. This
  reasoning paradigm significantly enhances the model's problem-solving abilities
  and has achieved promising results. However, long-thought reasoning process
  leads to a substantial increase in inference time. A pressing challenge is
  reducing the inference overhead of long-thought LLMs while ensuring accuracy.
  In this paper, we experimentally demonstrate that long-thought reasoning models
  struggle to effectively allocate token budgets based on problem difficulty and
  reasoning redundancies. To address this, we propose Length-Harmonizing
  Fine-Tuning (O1-Pruner), aiming at minimizing reasoning overhead while
  maintaining accuracy. This effective fine-tuning method first estimates the
  LLM's baseline performance through pre-sampling and then uses RL-style
  fine-tuning to encourage the model to generate shorter reasoning processes
  under accuracy constraints. This allows the model to achieve efficient
  reasoning with lower redundancy while maintaining accuracy. Experiments on
  various mathematical reasoning benchmarks show that O1-Pruner not only
  significantly reduces inference overhead but also achieves higher accuracy,
  providing a novel and promising solution to this challenge. Our code is coming
  soon at https://github.com/StarDewXXX/O1-Pruner
paper_authors._id: 6791b165c6fee5f0588773a9
paper_authors.name: Haotian Luo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-23T09:50:29.364Z
paper_authors._id: 674919d380981d58adbb456a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/q9n5VYYP7Ip6iPkrlDFGm.png
paper_authors.fullname: LuoHaotian
paper_authors.type: user
paper_authors.user: iNk233
paper_authors._id: 6791b165c6fee5f0588773aa
paper_authors.name: Li Shen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-26T11:38:40.905Z
paper_authors._id: 62de356ad89af8c07209e7d4
paper_authors.avatarUrl: /avatars/610629958726b270418368b8b7f61469.svg
paper_authors.fullname: Li Shen
paper_authors.type: user
paper_authors.user: mathshenli
paper_authors._id: 6791b165c6fee5f0588773ab
paper_authors.name: Haiying He
paper_authors._id: 6791b165c6fee5f0588773ac
paper_authors.name: Yibo Wang
paper_authors._id: 6791b165c6fee5f0588773ad
paper_authors.name: Shiwei Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-03T11:15:27.241Z
paper_authors._id: 65b04d2291e63920a7898c9e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/65b04d2291e63920a7898c9e/iUHs235G4bqK-KnH_94ti.jpeg
paper_authors.fullname: Liu
paper_authors.type: user
paper_authors.user: Shiweiliuiiiiiii
paper_authors._id: 6791b165c6fee5f0588773ae
paper_authors.name: Wei Li
paper_authors._id: 6791b165c6fee5f0588773af
paper_authors.name: Naiqiang Tan
paper_authors._id: 6791b165c6fee5f0588773b0
paper_authors.name: Xiaochun Cao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-23T09:50:53.299Z
paper_authors._id: 64ddc5ea83645d0f98599d6e
paper_authors.avatarUrl: /avatars/99a148c29cfbff77ed6150cc279a47c0.svg
paper_authors.fullname: caoxiaochun
paper_authors.type: user
paper_authors.user: cxc361461518
paper_authors._id: 6791b165c6fee5f0588773b1
paper_authors.name: Dacheng Tao
paper_title: O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning
paper_summary: Recently, long-thought reasoning LLMs, such as OpenAI's O1, adopt extended
  reasoning processes similar to how humans ponder over complex problems. This
  reasoning paradigm significantly enhances the model's problem-solving abilities
  and has achieved promising results. However, long-thought reasoning process
  leads to a substantial increase in inference time. A pressing challenge is
  reducing the inference overhead of long-thought LLMs while ensuring accuracy.
  In this paper, we experimentally demonstrate that long-thought reasoning models
  struggle to effectively allocate token budgets based on problem difficulty and
  reasoning redundancies. To address this, we propose Length-Harmonizing
  Fine-Tuning (O1-Pruner), aiming at minimizing reasoning overhead while
  maintaining accuracy. This effective fine-tuning method first estimates the
  LLM's baseline performance through pre-sampling and then uses RL-style
  fine-tuning to encourage the model to generate shorter reasoning processes
  under accuracy constraints. This allows the model to achieve efficient
  reasoning with lower redundancy while maintaining accuracy. Experiments on
  various mathematical reasoning benchmarks show that O1-Pruner not only
  significantly reduces inference overhead but also achieves higher accuracy,
  providing a novel and promising solution to this challenge. Our code is coming
  soon at https://github.com/StarDewXXX/O1-Pruner
paper_ai_keywords: sequence-to-sequence long-thought reasoning inference overhead token budgets reasoning redundancies Length-Harmonizing Fine-Tuning O1-Pruner pre-sampling RL-style fine-tuning