title: Scaling Reasoning, Losing Control: Evaluating Instruction Following in
    Large Reasoning Models
summary: Instruction-following is essential for aligning large language models (LLMs)
  with user intent. While recent reasoning-oriented models exhibit impressive
  performance on complex mathematical problems, their ability to adhere to
  natural language instructions remains underexplored. In this work, we introduce
  MathIF, a dedicated benchmark for evaluating instruction-following in
  mathematical reasoning tasks. Our empirical analysis reveals a consistent
  tension between scaling up reasoning capacity and maintaining controllability,
  as models that reason more effectively often struggle to comply with user
  directives. We find that models tuned on distilled long chains-of-thought or
  trained with reasoning-oriented reinforcement learning often degrade in
  instruction adherence, especially when generation length increases.
  Furthermore, we show that even simple interventions can partially recover
  obedience, though at the cost of reasoning performance. These findings
  highlight a fundamental tension in current LLM training paradigms and motivate
  the need for more instruction-aware reasoning models. We release the code and
  data at https://github.com/TingchenFu/MathIF.
paper_authors._id: 682ea2b450671dc82688b8ad
paper_authors.name: Tingchen Fu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-22T07:15:44.217Z
paper_authors._id: 640ad17a1ee054d66a74783e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/640ad17a1ee054d66a74783e/u0PjIkyC-9HkGzEyUQ7JN.jpeg
paper_authors.fullname: Tingchen Fu
paper_authors.type: user
paper_authors.user: TingchenFu
paper_authors._id: 682ea2b450671dc82688b8ae
paper_authors.name: Jiawei Gu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-26T08:21:57.596Z
paper_authors._id: 645b4819f9d4ec91fdd54852
paper_authors.avatarUrl: /avatars/e12efb8e030688a0afcc72176b453fb3.svg
paper_authors.fullname: Jiawei Gu
paper_authors.type: user
paper_authors.user: kuvvi
paper_authors._id: 682ea2b450671dc82688b8af
paper_authors.name: Yafu Li
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-05-22T04:06:13.396Z
paper_authors._id: 63f3502a520c14618925825a
paper_authors.avatarUrl: /avatars/e986a2a6625e7be6890616a417f908d2.svg
paper_authors.fullname: Yafu Li
paper_authors.type: user
paper_authors.user: yaful
paper_authors._id: 682ea2b450671dc82688b8b0
paper_authors.name: Xiaoye Qu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-26T08:21:28.583Z
paper_authors._id: 64cb54da1af278541d663708
paper_authors.avatarUrl: /avatars/c44507cc92bb2e83154bad31b90ce6dd.svg
paper_authors.fullname: Xiaoye Qu
paper_authors.type: user
paper_authors.user: Xiaoye08
paper_authors._id: 682ea2b450671dc82688b8b1
paper_authors.name: Yu Cheng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-02T07:49:34.787Z
paper_authors._id: 67017abfe4d49b157ac534d9
paper_authors.avatarUrl: /avatars/997e1b9f54b27a7728a9d4abfee4ba91.svg
paper_authors.fullname: Yu Cheng
paper_authors.type: user
paper_authors.user: ych133
paper_title: Scaling Reasoning, Losing Control: Evaluating Instruction Following in
    Large Reasoning Models
paper_summary: Instruction-following is essential for aligning large language models (LLMs)
  with user intent. While recent reasoning-oriented models exhibit impressive
  performance on complex mathematical problems, their ability to adhere to
  natural language instructions remains underexplored. In this work, we introduce
  MathIF, a dedicated benchmark for evaluating instruction-following in
  mathematical reasoning tasks. Our empirical analysis reveals a consistent
  tension between scaling up reasoning capacity and maintaining controllability,
  as models that reason more effectively often struggle to comply with user
  directives. We find that models tuned on distilled long chains-of-thought or
  trained with reasoning-oriented reinforcement learning often degrade in
  instruction adherence, especially when generation length increases.
  Furthermore, we show that even simple interventions can partially recover
  obedience, though at the cost of reasoning performance. These findings
  highlight a fundamental tension in current LLM training paradigms and motivate
  the need for more instruction-aware reasoning models. We release the code and
  data at https://github.com/TingchenFu/MathIF.
paper_ai_summary: An empirical analysis of MathIF identifies a tension between enhancing reasoning capacity and maintaining instruction adherence in large language models.
paper_ai_keywords: instruction-following reasoning-oriented models benchmarks chains-of-thought reinforcement learning instruction adherence