title: Walk Before You Run! Concise LLM Reasoning via Reinforcement Learning
summary: As test-time scaling becomes a pivotal research frontier in Large Language
  Models (LLMs) development, contemporary and advanced post-training
  methodologies increasingly focus on extending the generation length of long
  Chain-of-Thought (CoT) responses to enhance reasoning capabilities toward
  DeepSeek R1-like performance. However, recent studies reveal a persistent
  overthinking phenomenon in state-of-the-art reasoning models, manifesting as
  excessive redundancy or repetitive thinking patterns in long CoT responses. To
  address this issue, in this paper, we propose a simple yet effective two-stage
  reinforcement learning framework for achieving concise reasoning in LLMs, named
  ConciseR. Specifically, the first stage, using more training steps, aims to
  incentivize the model's reasoning capabilities via Group Relative Policy
  Optimization with clip-higher and dynamic sampling components (GRPO++), and the
  second stage, using fewer training steps, explicitly enforces conciseness and
  improves efficiency via Length-aware Group Relative Policy Optimization
  (L-GRPO). Significantly, ConciseR only optimizes response length once all
  rollouts of a sample are correct, following the "walk before you run"
  principle. Extensive experimental results demonstrate that our ConciseR model,
  which generates more concise CoT reasoning responses, outperforms recent
  state-of-the-art reasoning models with zero RL paradigm across AIME 2024,
  MATH-500, AMC 2023, Minerva, and Olympiad benchmarks.
paper_authors._id: 6836b5fccbd5554d2038732c
paper_authors.name: Mingyang Song
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-28T08:55:33.180Z
paper_authors._id: 617051728db4a760d912d81f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/617051728db4a760d912d81f/4dKG176GPk2pPRK7ctaT-.jpeg
paper_authors.fullname: Mingyang Song
paper_authors.type: user
paper_authors.user: Nickyang
paper_authors._id: 6836b5fccbd5554d2038732d
paper_authors.name: Mao Zheng
paper_title: Walk Before You Run! Concise LLM Reasoning via Reinforcement Learning
paper_summary: As test-time scaling becomes a pivotal research frontier in Large Language
  Models (LLMs) development, contemporary and advanced post-training
  methodologies increasingly focus on extending the generation length of long
  Chain-of-Thought (CoT) responses to enhance reasoning capabilities toward
  DeepSeek R1-like performance. However, recent studies reveal a persistent
  overthinking phenomenon in state-of-the-art reasoning models, manifesting as
  excessive redundancy or repetitive thinking patterns in long CoT responses. To
  address this issue, in this paper, we propose a simple yet effective two-stage
  reinforcement learning framework for achieving concise reasoning in LLMs, named
  ConciseR. Specifically, the first stage, using more training steps, aims to
  incentivize the model's reasoning capabilities via Group Relative Policy
  Optimization with clip-higher and dynamic sampling components (GRPO++), and the
  second stage, using fewer training steps, explicitly enforces conciseness and
  improves efficiency via Length-aware Group Relative Policy Optimization
  (L-GRPO). Significantly, ConciseR only optimizes response length once all
  rollouts of a sample are correct, following the "walk before you run"
  principle. Extensive experimental results demonstrate that our ConciseR model,
  which generates more concise CoT reasoning responses, outperforms recent
  state-of-the-art reasoning models with zero RL paradigm across AIME 2024,
  MATH-500, AMC 2023, Minerva, and Olympiad benchmarks.
paper_ai_summary: A reinforcement learning framework, ConciseR, is proposed to enhance the conciseness and efficiency of reasoning in LLMs through a two-stage optimization process.
paper_ai_keywords: Large Language Models Chain-of-Thought DeepSeek R1 reinforcement learning Group Relative Policy Optimization clip-higher dynamic sampling Length-aware Group Relative Policy Optimization concise reasoning rollouts AIME 2024 MATH-500 AMC 2023 Minerva Olympiad benchmarks