title: R1-Searcher: Incentivizing the Search Capability in LLMs via
    Reinforcement Learning
summary: Existing Large Reasoning Models (LRMs) have shown the potential of
  reinforcement learning (RL) to enhance the complex reasoning capabilities of
  Large Language Models~(LLMs). While they achieve remarkable performance on
  challenging tasks such as mathematics and coding, they often rely on their
  internal knowledge to solve problems, which can be inadequate for
  time-sensitive or knowledge-intensive questions, leading to inaccuracies and
  hallucinations. To address this, we propose R1-Searcher, a novel
  two-stage outcome-based RL approach designed to enhance the search capabilities
  of LLMs. This method allows LLMs to autonomously invoke external search systems
  to access additional knowledge during the reasoning process. Our framework
  relies exclusively on RL, without requiring process rewards or distillation for
  a cold start. % effectively generalizing to out-of-domain datasets and
  supporting both Base and Instruct models. Our experiments demonstrate that our
  method significantly outperforms previous strong RAG methods, even when
  compared to the closed-source GPT-4o-mini.
paper_authors._id: 67ce5fd2e5cdfda52b9123a4
paper_authors.name: Huatong Song
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-10T10:03:49.730Z
paper_authors._id: 66163dc8c7f45b3f893ff40b
paper_authors.avatarUrl: /avatars/801043dac0caae90bbca8c9d3e2e203b.svg
paper_authors.fullname: Song Huatong
paper_authors.type: user
paper_authors.user: XXsongLALA
paper_authors._id: 67ce5fd2e5cdfda52b9123a5
paper_authors.name: Jinhao Jiang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-10T10:04:22.446Z
paper_authors._id: 61b8405b516a20acdf3b85ff
paper_authors.avatarUrl: /avatars/3d2eae7c163a80b73260087b05a4230b.svg
paper_authors.fullname: Jinhao Jiang
paper_authors.type: user
paper_authors.user: Boru
paper_authors._id: 67ce5fd2e5cdfda52b9123a6
paper_authors.name: Yingqian Min
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-10T09:40:54.171Z
paper_authors._id: 6703ac76ea890f0ca5b225eb
paper_authors.avatarUrl: /avatars/5f56c49a1940143d47dd484782a4abbf.svg
paper_authors.fullname: Yingqian Min
paper_authors.type: user
paper_authors.user: EliverQ
paper_authors._id: 67ce5fd2e5cdfda52b9123a7
paper_authors.name: Jie Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-10T12:46:26.714Z
paper_authors._id: 651a29d566e78720a78317ec
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/651a29d566e78720a78317ec/icoHA_v1RPVfwTooN3fJb.png
paper_authors.fullname: Jie Chen
paper_authors.type: user
paper_authors.user: survivi
paper_authors._id: 67ce5fd2e5cdfda52b9123a8
paper_authors.name: Zhipeng Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-10T11:08:49.613Z
paper_authors._id: 629b765ce1af194c641fcbc6
paper_authors.avatarUrl: /avatars/7c53a4c2a1e528c19641a2b601731754.svg
paper_authors.fullname: Zhipeng Chen
paper_authors.type: user
paper_authors.user: TimothyCzp
paper_authors._id: 67ce5fd2e5cdfda52b9123a9
paper_authors.name: Wayne Xin Zhao
paper_authors._id: 67ce5fd2e5cdfda52b9123aa
paper_authors.name: Lei Fang
paper_authors._id: 67ce5fd2e5cdfda52b9123ab
paper_authors.name: Ji-Rong Wen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-10T10:04:33.194Z
paper_authors._id: 64b8c89052b7353d8c6a1013
paper_authors.avatarUrl: /avatars/cd59fffe81f6b07b4519540b8ff3d95f.svg
paper_authors.fullname: Ji-Rong Wen
paper_authors.type: user
paper_authors.user: jrwen
paper_title: R1-Searcher: Incentivizing the Search Capability in LLMs via
    Reinforcement Learning
paper_summary: Existing Large Reasoning Models (LRMs) have shown the potential of
  reinforcement learning (RL) to enhance the complex reasoning capabilities of
  Large Language Models~(LLMs). While they achieve remarkable performance on
  challenging tasks such as mathematics and coding, they often rely on their
  internal knowledge to solve problems, which can be inadequate for
  time-sensitive or knowledge-intensive questions, leading to inaccuracies and
  hallucinations. To address this, we propose R1-Searcher, a novel
  two-stage outcome-based RL approach designed to enhance the search capabilities
  of LLMs. This method allows LLMs to autonomously invoke external search systems
  to access additional knowledge during the reasoning process. Our framework
  relies exclusively on RL, without requiring process rewards or distillation for
  a cold start. % effectively generalizing to out-of-domain datasets and
  supporting both Base and Instruct models. Our experiments demonstrate that our
  method significantly outperforms previous strong RAG methods, even when
  compared to the closed-source GPT-4o-mini.
paper_ai_keywords: reinforcement learning (RL) Large Reasoning Models (LRMs) Large Language Models (LLMs) two-stage outcome-based RL external search systems R1-Searcher