title: Incentivizing Reasoning for Advanced Instruction-Following of Large
    Language Models
summary: Existing large language models (LLMs) face challenges of following complex
  instructions, especially when multiple constraints are present and organized in
  paralleling, chaining, and branching structures. One intuitive solution, namely
  chain-of-thought (CoT), is expected to universally improve capabilities of
  LLMs. However, we find that the vanilla CoT exerts a negative impact on
  performance due to its superficial reasoning pattern of simply paraphrasing the
  instructions. It fails to peel back the compositions of constraints for
  identifying their relationship across hierarchies of types and dimensions. To
  this end, we propose a systematic method to boost LLMs in dealing with complex
  instructions via incentivizing reasoning for test-time compute scaling. First,
  we stem from the decomposition of complex instructions under existing
  taxonomies and propose a reproducible data acquisition method. Second, we
  exploit reinforcement learning (RL) with verifiable rule-centric reward signals
  to cultivate reasoning specifically for instruction following. We address the
  shallow, non-essential nature of reasoning under complex instructions via
  sample-wise contrast for superior CoT enforcement. We also exploit behavior
  cloning of experts to facilitate steady distribution shift from fast-thinking
  LLMs to skillful reasoners. Extensive evaluations on seven comprehensive
  benchmarks confirm the validity of the proposed method, where a 1.5B LLM
  achieves 11.74% gains with performance comparable to a 8B LLM. Codes and data
  are available at https://github.com/yuleiqin/RAIF.
paper_authors._id: 683e6aa057738c5cc3616d70
paper_authors.name: Yulei Qin
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-06-03T03:23:18.288Z
paper_authors._id: 6390525c00fb8ec4a424e0ff
paper_authors.avatarUrl: /avatars/4302571e2ef4a9875491221aa630a329.svg
paper_authors.fullname: Yulei Qin
paper_authors.type: user
paper_authors.user: yolay
paper_authors._id: 683e6aa057738c5cc3616d71
paper_authors.name: Gang Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-03T12:33:53.264Z
paper_authors._id: 648c17fc17c7ceb9b4200d31
paper_authors.avatarUrl: /avatars/abc5b025dd682935056c33a9e1e362f0.svg
paper_authors.fullname: Gang Li
paper_authors.type: user
paper_authors.user: ligang-cs
paper_authors._id: 683e6aa057738c5cc3616d72
paper_authors.name: Zongyi Li
paper_authors._id: 683e6aa057738c5cc3616d73
paper_authors.name: Zihan Xu
paper_authors._id: 683e6aa057738c5cc3616d74
paper_authors.name: Yuchen Shi
paper_authors._id: 683e6aa057738c5cc3616d75
paper_authors.name: Zhekai Lin
paper_authors._id: 683e6aa057738c5cc3616d76
paper_authors.name: Xiao Cui
paper_authors._id: 683e6aa057738c5cc3616d77
paper_authors.name: Ke Li
paper_authors._id: 683e6aa057738c5cc3616d78
paper_authors.name: Xing Sun
paper_title: Incentivizing Reasoning for Advanced Instruction-Following of Large
    Language Models
paper_summary: Existing large language models (LLMs) face challenges of following complex
  instructions, especially when multiple constraints are present and organized in
  paralleling, chaining, and branching structures. One intuitive solution, namely
  chain-of-thought (CoT), is expected to universally improve capabilities of
  LLMs. However, we find that the vanilla CoT exerts a negative impact on
  performance due to its superficial reasoning pattern of simply paraphrasing the
  instructions. It fails to peel back the compositions of constraints for
  identifying their relationship across hierarchies of types and dimensions. To
  this end, we propose a systematic method to boost LLMs in dealing with complex
  instructions via incentivizing reasoning for test-time compute scaling. First,
  we stem from the decomposition of complex instructions under existing
  taxonomies and propose a reproducible data acquisition method. Second, we
  exploit reinforcement learning (RL) with verifiable rule-centric reward signals
  to cultivate reasoning specifically for instruction following. We address the
  shallow, non-essential nature of reasoning under complex instructions via
  sample-wise contrast for superior CoT enforcement. We also exploit behavior
  cloning of experts to facilitate steady distribution shift from fast-thinking
  LLMs to skillful reasoners. Extensive evaluations on seven comprehensive
  benchmarks confirm the validity of the proposed method, where a 1.5B LLM
  achieves 11.74% gains with performance comparable to a 8B LLM. Codes and data
  are available at https://github.com/yuleiqin/RAIF.
paper_ai_summary: A method is proposed to enhance large language models in handling complex instructions through incentivized reasoning and reinforcement learning, improving performance and reducing computational load.
paper_ai_keywords: chain-of-thought (CoT) reinforcement learning (RL) rule-centric reward signals sample-wise contrast behavior cloning instruction following decomposition of complex instructions