R1-Searcher: Incentivizing the Search Capability in LLMs via
  Reinforcement Learning
Existing Large Reasoning Models (LRMs) have shown the potential of
reinforcement learning (RL) to enhance the complex reasoning capabilities of
Large Language Models~(LLMs). While they achieve remarkable performance on
challenging tasks such as mathematics and coding, they often rely on their
internal knowledge to solve problems, which can be inadequate for
time-sensitive or knowledge-intensive questions, leading to inaccuracies and
hallucinations. To address this, we propose R1-Searcher, a novel
two-stage outcome-based RL approach designed to enhance the search capabilities
of LLMs. This method allows LLMs to autonomously invoke external search systems
to access additional knowledge during the reasoning process. Our framework
relies exclusively on RL, without requiring process rewards or distillation for
a cold start. % effectively generalizing to out-of-domain datasets and
supporting both Base and Instruct models. Our experiments demonstrate that our
method significantly outperforms previous strong RAG methods, even when
compared to the closed-source GPT-4o-mini.
67ce5fd2e5cdfda52b9123a4
Huatong Song
admin_assigned
2025-03-10T10:03:49.730Z
66163dc8c7f45b3f893ff40b
/avatars/801043dac0caae90bbca8c9d3e2e203b.svg
Song Huatong
user
XXsongLALA
67ce5fd2e5cdfda52b9123a5
Jinhao Jiang
admin_assigned
2025-03-10T10:04:22.446Z
61b8405b516a20acdf3b85ff
/avatars/3d2eae7c163a80b73260087b05a4230b.svg
Jinhao Jiang
user
Boru
67ce5fd2e5cdfda52b9123a6
Yingqian Min
claimed_verified
2025-03-10T09:40:54.171Z
6703ac76ea890f0ca5b225eb
/avatars/5f56c49a1940143d47dd484782a4abbf.svg
Yingqian Min
user
EliverQ
67ce5fd2e5cdfda52b9123a7
Jie Chen
claimed_verified
2025-03-10T12:46:26.714Z
651a29d566e78720a78317ec
https://cdn-avatars.huggingface.co/v1/production/uploads/651a29d566e78720a78317ec/8yTNvYeUs8cZo5GWvPPOJ.jpeg
Jie Chen
user
survivi
67ce5fd2e5cdfda52b9123a8
Zhipeng Chen
claimed_verified
2025-03-10T11:08:49.613Z
629b765ce1af194c641fcbc6
/avatars/7c53a4c2a1e528c19641a2b601731754.svg
Zhipeng Chen
user
TimothyCzp
67ce5fd2e5cdfda52b9123a9
Wayne Xin Zhao
67ce5fd2e5cdfda52b9123aa
Lei Fang
67ce5fd2e5cdfda52b9123ab
Ji-Rong Wen
admin_assigned
2025-03-10T10:04:33.194Z
64b8c89052b7353d8c6a1013
/avatars/cd59fffe81f6b07b4519540b8ff3d95f.svg
Ji-Rong Wen
user
jrwen
R1-Searcher: Incentivizing the Search Capability in LLMs via
  Reinforcement Learning
Existing Large Reasoning Models (LRMs) have shown the potential of
reinforcement learning (RL) to enhance the complex reasoning capabilities of
Large Language Models~(LLMs). While they achieve remarkable performance on
challenging tasks such as mathematics and coding, they often rely on their
internal knowledge to solve problems, which can be inadequate for
time-sensitive or knowledge-intensive questions, leading to inaccuracies and
hallucinations. To address this, we propose R1-Searcher, a novel
two-stage outcome-based RL approach designed to enhance the search capabilities
of LLMs. This method allows LLMs to autonomously invoke external search systems
to access additional knowledge during the reasoning process. Our framework
relies exclusively on RL, without requiring process rewards or distillation for
a cold start. % effectively generalizing to out-of-domain datasets and
supporting both Base and Instruct models. Our experiments demonstrate that our
method significantly outperforms previous strong RAG methods, even when
compared to the closed-source GPT-4o-mini.
R1-Searcher is a reinforcement learning approach that enhances large language models' reasoning by autonomously accessing external knowledge, achieving better performance than existing methods.
reinforcement learning LLMs LRM R1-Searcher two-stage outcome-based RL external search systems RL RAG methods GPT-4o-mini