title: Search-R1: Training LLMs to Reason and Leverage Search Engines with
    Reinforcement Learning
summary: Efficiently acquiring external knowledge and up-to-date information is
  essential for effective reasoning and text generation in large language models
  (LLMs). Retrieval augmentation and tool-use training approaches where a search
  engine is treated as a tool lack complex multi-turn retrieval flexibility or
  require large-scale supervised data. Prompting advanced LLMs with reasoning
  capabilities during inference to use search engines is not optimal, since the
  LLM does not learn how to optimally interact with the search engine. This paper
  introduces Search-R1, an extension of the DeepSeek-R1 model where the LLM
  learns -- solely through reinforcement learning (RL) -- to autonomously
  generate (multiple) search queries during step-by-step reasoning with real-time
  retrieval. Search-R1 optimizes LLM rollouts with multi-turn search
  interactions, leveraging retrieved token masking for stable RL training and a
  simple outcome-based reward function. Experiments on seven question-answering
  datasets show that Search-R1 improves performance by 26% (Qwen2.5-7B), 21%
  (Qwen2.5-3B), and 10% (LLaMA3.2-3B) over SOTA baselines. This paper further
  provides empirical insights into RL optimization methods, LLM choices, and
  response length dynamics in retrieval-augmented reasoning. The code and model
  checkpoints are available at https://github.com/PeterGriffinJin/Search-R1.
paper_authors._id: 67d238ab7d0fc37e671feaa8
paper_authors.name: Bowen Jin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-13T08:24:22.835Z
paper_authors._id: 64b8222749bde5d948104627
paper_authors.avatarUrl: /avatars/e179dc68aebae503dcd7ea6b65b4a4b7.svg
paper_authors.fullname: Bowen
paper_authors.type: user
paper_authors.user: PeterJinGo
paper_authors._id: 67d238ab7d0fc37e671feaa9
paper_authors.name: Hansi Zeng
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-14T11:24:04.316Z
paper_authors._id: 64395fa0d45e8db3e28667a4
paper_authors.avatarUrl: /avatars/cd050f59e2ac79f02ee01dcbb3be00e5.svg
paper_authors.fullname: Hansi Zeng
paper_authors.type: user
paper_authors.user: hzeng
paper_authors._id: 67d238ab7d0fc37e671feaaa
paper_authors.name: Zhenrui Yue
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-03-14T11:24:10.545Z
paper_authors._id: 644c4eff9a7c19cfaf0c8f4a
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/2gHtfhPFjSsnwxVtBnxe2.jpeg
paper_authors.fullname: Zhenrui Yue
paper_authors.type: user
paper_authors.user: yuezrhb
paper_authors._id: 67d238ab7d0fc37e671feaab
paper_authors.name: Dong Wang
paper_authors._id: 67d238ab7d0fc37e671feaac
paper_authors.name: Hamed Zamani
paper_authors._id: 67d238ab7d0fc37e671feaad
paper_authors.name: Jiawei Han
paper_title: Search-R1: Training LLMs to Reason and Leverage Search Engines with
    Reinforcement Learning
paper_summary: Efficiently acquiring external knowledge and up-to-date information is
  essential for effective reasoning and text generation in large language models
  (LLMs). Retrieval augmentation and tool-use training approaches where a search
  engine is treated as a tool lack complex multi-turn retrieval flexibility or
  require large-scale supervised data. Prompting advanced LLMs with reasoning
  capabilities during inference to use search engines is not optimal, since the
  LLM does not learn how to optimally interact with the search engine. This paper
  introduces Search-R1, an extension of the DeepSeek-R1 model where the LLM
  learns -- solely through reinforcement learning (RL) -- to autonomously
  generate (multiple) search queries during step-by-step reasoning with real-time
  retrieval. Search-R1 optimizes LLM rollouts with multi-turn search
  interactions, leveraging retrieved token masking for stable RL training and a
  simple outcome-based reward function. Experiments on seven question-answering
  datasets show that Search-R1 improves performance by 26% (Qwen2.5-7B), 21%
  (Qwen2.5-3B), and 10% (LLaMA3.2-3B) over SOTA baselines. This paper further
  provides empirical insights into RL optimization methods, LLM choices, and
  response length dynamics in retrieval-augmented reasoning. The code and model
  checkpoints are available at https://github.com/PeterGriffinJin/Search-R1.
paper_ai_summary: Search-R1, an extension of DeepSeek-R1, enhances large language models by autonomously generating multiple search queries during step-by-step reasoning using reinforcement learning, improving performance in retrieval-augmented question-answering tasks.
paper_ai_keywords: retrieval augmentation reinforcement learning search queries step-by-step reasoning real-time retrieval outcome-based reward function multi-turn search interactions token masking