title: ReasonFlux: Hierarchical LLM Reasoning via Scaling Thought Templates
summary: We present that hierarchical LLM reasoning via scaling thought templates can
  effectively optimize the reasoning search space and outperform the mathematical
  reasoning capabilities of powerful LLMs like OpenAI o1-preview and DeepSeek V3.
  We train our ReasonFlux-32B model with only 8 GPUs and introduces three
  innovations: (i) a structured and generic thought template library, containing
  around 500 high-level thought templates capable of generalizing to similar or
  relevant reasoning problems; (ii) performing hierarchical reinforcement
  learning on a sequence of thought templates instead of long CoTs, optimizing a
  base LLM to plan out an optimal template trajectory for gradually handling
  complex problems; (iii) a brand new inference scaling system that enables
  hierarchical LLM reasoning by adaptively scaling thought templates at inference
  time. With a template trajectory containing sequential thought templates, our
  ReasonFlux-32B significantly advances math reasoning capabilities to
  state-of-the-art levels. Notably, on the MATH benchmark, it achieves an
  accuracy of 91.2% and surpasses o1-preview by 6.7%. On the USA Math Olympiad
  (AIME) benchmark, ReasonFlux-32B solves an average of 56.7% of problems,
  surpassing o1-preview and DeepSeek-V3 by 27% and 45%, respectively. Code:
  https://github.com/Gen-Verse/ReasonFlux
paper_authors._id: 67aac8adfe33f6d8d695bc40
paper_authors.name: Ling Yang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-11T14:25:31.970Z
paper_authors._id: 64fde4e252e82dd432b74ce9
paper_authors.avatarUrl: /avatars/061a69d858b86d1600be916122cae7fc.svg
paper_authors.fullname: Ling Yang
paper_authors.type: user
paper_authors.user: Lingaaaaaaa
paper_authors._id: 67aac8adfe33f6d8d695bc41
paper_authors.name: Zhaochen Yu
paper_authors._id: 67aac8adfe33f6d8d695bc42
paper_authors.name: Bin Cui
paper_authors._id: 67aac8adfe33f6d8d695bc43
paper_authors.name: Mengdi Wang
paper_title: ReasonFlux: Hierarchical LLM Reasoning via Scaling Thought Templates
paper_summary: We present that hierarchical LLM reasoning via scaling thought templates can
  effectively optimize the reasoning search space and outperform the mathematical
  reasoning capabilities of powerful LLMs like OpenAI o1-preview and DeepSeek V3.
  We train our ReasonFlux-32B model with only 8 GPUs and introduces three
  innovations: (i) a structured and generic thought template library, containing
  around 500 high-level thought templates capable of generalizing to similar or
  relevant reasoning problems; (ii) performing hierarchical reinforcement
  learning on a sequence of thought templates instead of long CoTs, optimizing a
  base LLM to plan out an optimal template trajectory for gradually handling
  complex problems; (iii) a brand new inference scaling system that enables
  hierarchical LLM reasoning by adaptively scaling thought templates at inference
  time. With a template trajectory containing sequential thought templates, our
  ReasonFlux-32B significantly advances math reasoning capabilities to
  state-of-the-art levels. Notably, on the MATH benchmark, it achieves an
  accuracy of 91.2% and surpasses o1-preview by 6.7%. On the USA Math Olympiad
  (AIME) benchmark, ReasonFlux-32B solves an average of 56.7% of problems,
  surpassing o1-preview and DeepSeek-V3 by 27% and 45%, respectively. Code:
  https://github.com/Gen-Verse/ReasonFlux
paper_ai_summary: Hierarchical reasoning with LLMs using scaled thought templates improves mathematical reasoning and outperforms existing models on benchmarks like MATH and AIME.
paper_ai_keywords: hierarchical LLM reasoning thought templates mathematical reasoning reinforcement learning template trajectory inference scaling system ReasonFlux-32B MATH benchmark USA Math Olympiad AIME benchmark