title: Towards Large Reasoning Models: A Survey of Reinforced Reasoning with
    Large Language Models
summary: Language has long been conceived as an essential tool for human reasoning.
  The breakthrough of Large Language Models (LLMs) has sparked significant
  research interest in leveraging these models to tackle complex reasoning tasks.
  Researchers have moved beyond simple autoregressive token generation by
  introducing the concept of "thought" -- a sequence of tokens representing
  intermediate steps in the reasoning process. This innovative paradigm enables
  LLMs' to mimic complex human reasoning processes, such as tree search and
  reflective thinking. Recently, an emerging trend of learning to reason has
  applied reinforcement learning (RL) to train LLMs to master reasoning
  processes. This approach enables the automatic generation of high-quality
  reasoning trajectories through trial-and-error search algorithms, significantly
  expanding LLMs' reasoning capacity by providing substantially more training
  data. Furthermore, recent studies demonstrate that encouraging LLMs to "think"
  with more tokens during test-time inference can further significantly boost
  reasoning accuracy. Therefore, the train-time and test-time scaling combined to
  show a new research frontier -- a path toward Large Reasoning Model. The
  introduction of OpenAI's o1 series marks a significant milestone in this
  research direction. In this survey, we present a comprehensive review of recent
  progress in LLM reasoning. We begin by introducing the foundational background
  of LLMs and then explore the key technical components driving the development
  of large reasoning models, with a focus on automated data construction,
  learning-to-reason techniques, and test-time scaling. We also analyze popular
  open-source projects at building large reasoning models, and conclude with open
  challenges and future research directions.
paper_authors._id: 6789e04652739943c940af38
paper_authors.name: Fengli Xu
paper_authors._id: 6789e04652739943c940af39
paper_authors.name: Qianyue Hao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-17T10:42:15.441Z
paper_authors._id: 6583e2b283a9e1460c6fb1e0
paper_authors.avatarUrl: /avatars/a949165b1cec5e1d1d55f3af98182156.svg
paper_authors.fullname: Qianyue Hao
paper_authors.type: user
paper_authors.user: haohao11
paper_authors._id: 6789e04652739943c940af3a
paper_authors.name: Zefang Zong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-17T10:42:26.065Z
paper_authors._id: 64feba7efa64465422ce3003
paper_authors.avatarUrl: /avatars/abdc7a3748f6a4e15ebc6aa8d616c87d.svg
paper_authors.fullname: zongzefang
paper_authors.type: user
paper_authors.user: zzfoutofspace
paper_authors._id: 6789e04652739943c940af3b
paper_authors.name: Jingwei Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-30T08:41:20.675Z
paper_authors._id: 66e59ff27d43e55cfd31bec9
paper_authors.avatarUrl: /avatars/e046b51bf2a530255d3fb707991546a4.svg
paper_authors.fullname: Chen Gao
paper_authors.type: user
paper_authors.user: chgao96
paper_authors._id: 6789e04652739943c940af3c
paper_authors.name: Yunke Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-17T10:43:05.430Z
paper_authors._id: 64071cbc2e309e65451f87b6
paper_authors.avatarUrl: /avatars/2201244380d221b9db5661f20510d853.svg
paper_authors.fullname: yunke zhang
paper_authors.type: user
paper_authors.user: berserkerko
paper_authors._id: 6789e04652739943c940af3d
paper_authors.name: Jingyi Wang
paper_authors._id: 6789e04652739943c940af3e
paper_authors.name: Xiaochong Lan
paper_authors._id: 6789e04652739943c940af3f
paper_authors.name: Jiahui Gong
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-17T10:43:44.415Z
paper_authors._id: 6509a040d95f30b9dcdbf789
paper_authors.avatarUrl: /avatars/4e4b441e22a1f2c3387e5b981ba6bbbe.svg
paper_authors.fullname: Jiahui Gong
paper_authors.type: user
paper_authors.user: zhazhahui7
paper_authors._id: 6789e04652739943c940af40
paper_authors.name: Tianjian Ouyang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-17T10:43:50.599Z
paper_authors._id: 6566fcb5118497d0af91dc3b
paper_authors.avatarUrl: /avatars/e081d7d1b1657245cd818e5417cdcb2e.svg
paper_authors.fullname: TIANJIAN OUYANG
paper_authors.type: user
paper_authors.user: Ouyangtj
paper_authors._id: 6789e04652739943c940af41
paper_authors.name: Fanjin Meng
paper_authors._id: 6789e04652739943c940af42
paper_authors.name: Chenyang Shao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-17T10:44:14.869Z
paper_authors._id: 654dd731671d2c1ced0539e1
paper_authors.avatarUrl: /avatars/7eaa5eee16aac9105cd2af7ee84841c3.svg
paper_authors.fullname: shaochenyang
paper_authors.type: user
paper_authors.user: l-1-l
paper_authors._id: 6789e04652739943c940af43
paper_authors.name: Yuwei Yan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-17T10:44:24.670Z
paper_authors._id: 668e965959d9ffef7e8b035a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/tqjvWWxXgTkNhYeNhoPJQ.jpeg
paper_authors.fullname: Yuwei Yan
paper_authors.type: user
paper_authors.user: PinkGranite
paper_authors._id: 6789e04652739943c940af44
paper_authors.name: Qinglong Yang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-17T10:44:31.561Z
paper_authors._id: 663ef727f9c2d3c9da3b8d4b
paper_authors.avatarUrl: /avatars/4bb431e410078f7ba2487597d1beb589.svg
paper_authors.fullname: qinglong yang
paper_authors.type: user
paper_authors.user: m912218831
paper_authors._id: 6789e04652739943c940af45
paper_authors.name: Yiwen Song
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-17T10:44:57.348Z
paper_authors._id: 6517326e335b4c728b3406a4
paper_authors.avatarUrl: /avatars/cde4a672778fc69c172e6bdeee2d540a.svg
paper_authors.fullname: Yiwen Song
paper_authors.type: user
paper_authors.user: yiwen-song
paper_authors._id: 6789e04652739943c940af46
paper_authors.name: Sijian Ren
paper_authors._id: 6789e04652739943c940af47
paper_authors.name: Xinyuan Hu
paper_authors._id: 6789e04652739943c940af48
paper_authors.name: Yu Li
paper_authors._id: 6789e04652739943c940af49
paper_authors.name: Jie Feng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-20T09:29:08.372Z
paper_authors._id: 6465d3bd63e7e09dd02e95c3
paper_authors.avatarUrl: /avatars/b2798bd5f8368f956bf7fab79d9432f0.svg
paper_authors.fullname: Jie
paper_authors.type: user
paper_authors.user: JJ-TMT
paper_authors._id: 6789e04652739943c940af4a
paper_authors.name: Chen Gao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-17T10:45:59.888Z
paper_authors._id: 65049f41c6ae3df8f28cfe96
paper_authors.avatarUrl: /avatars/a653ed4a215c4bd34710a1cee9f4d9cd.svg
paper_authors.fullname: Chen Gao
paper_authors.type: user
paper_authors.user: gaochen315
paper_authors._id: 6789e04652739943c940af4b
paper_authors.name: Yong Li
paper_title: Towards Large Reasoning Models: A Survey of Reinforced Reasoning with
    Large Language Models
paper_summary: Language has long been conceived as an essential tool for human reasoning.
  The breakthrough of Large Language Models (LLMs) has sparked significant
  research interest in leveraging these models to tackle complex reasoning tasks.
  Researchers have moved beyond simple autoregressive token generation by
  introducing the concept of "thought" -- a sequence of tokens representing
  intermediate steps in the reasoning process. This innovative paradigm enables
  LLMs' to mimic complex human reasoning processes, such as tree search and
  reflective thinking. Recently, an emerging trend of learning to reason has
  applied reinforcement learning (RL) to train LLMs to master reasoning
  processes. This approach enables the automatic generation of high-quality
  reasoning trajectories through trial-and-error search algorithms, significantly
  expanding LLMs' reasoning capacity by providing substantially more training
  data. Furthermore, recent studies demonstrate that encouraging LLMs to "think"
  with more tokens during test-time inference can further significantly boost
  reasoning accuracy. Therefore, the train-time and test-time scaling combined to
  show a new research frontier -- a path toward Large Reasoning Model. The
  introduction of OpenAI's o1 series marks a significant milestone in this
  research direction. In this survey, we present a comprehensive review of recent
  progress in LLM reasoning. We begin by introducing the foundational background
  of LLMs and then explore the key technical components driving the development
  of large reasoning models, with a focus on automated data construction,
  learning-to-reason techniques, and test-time scaling. We also analyze popular
  open-source projects at building large reasoning models, and conclude with open
  challenges and future research directions.
paper_ai_keywords: Large Language Models (LLMs) autoregressive token generation thought tree search reflective thinking reinforcement learning (RL) reasoning trajectories trial-and-error search algorithms train-time scaling test-time scaling Large Reasoning Model OpenAI's o1 series automated data construction learning-to-reason techniques test-time scaling