title: ReSearch: Learning to Reason with Search for LLMs via Reinforcement
    Learning
summary: Large Language Models (LLMs) have shown remarkable capabilities in reasoning,
  exemplified by the success of OpenAI-o1 and DeepSeek-R1. However, integrating
  reasoning with external search processes remains challenging, especially for
  complex multi-hop questions requiring multiple retrieval steps. We propose
  ReSearch, a novel framework that trains LLMs to Reason with Search via
  reinforcement learning without using any supervised data on reasoning steps.
  Our approach treats search operations as integral components of the reasoning
  chain, where when and how to perform searches is guided by text-based thinking,
  and search results subsequently influence further reasoning. We train ReSearch
  on Qwen2.5-7B(-Instruct) and Qwen2.5-32B(-Instruct) models and conduct
  extensive experiments. Despite being trained on only one dataset, our models
  demonstrate strong generalizability across various benchmarks. Analysis reveals
  that ReSearch naturally elicits advanced reasoning capabilities such as
  reflection and self-correction during the reinforcement learning process.
paper_authors._id: 67e365b0dcfc2aeae1bf3da2
paper_authors.name: Mingyang Chen
paper_authors._id: 67e365b0dcfc2aeae1bf3da3
paper_authors.name: Tianpeng Li
paper_authors._id: 67e365b0dcfc2aeae1bf3da4
paper_authors.name: Haoze Sun
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-15T10:41:04.653Z
paper_authors._id: 6436bb0dd58a5ea528c55acb
paper_authors.avatarUrl: /avatars/df17b66780e14e07bbe4625f068a94ad.svg
paper_authors.fullname: Alvin Sun
paper_authors.type: user
paper_authors.user: AlvinSunYooo
paper_authors._id: 67e365b0dcfc2aeae1bf3da5
paper_authors.name: Yijie Zhou
paper_authors._id: 67e365b0dcfc2aeae1bf3da6
paper_authors.name: Chenzheng Zhu
paper_authors._id: 67e365b0dcfc2aeae1bf3da7
paper_authors.name: Fan Yang
paper_authors._id: 67e365b0dcfc2aeae1bf3da8
paper_authors.name: Zenan Zhou
paper_authors._id: 67e365b0dcfc2aeae1bf3da9
paper_authors.name: Weipeng Chen
paper_authors._id: 67e365b0dcfc2aeae1bf3daa
paper_authors.name: Haofen Wang
paper_authors._id: 67e365b0dcfc2aeae1bf3dab
paper_authors.name: Jeff Z. Pan
paper_authors._id: 67e365b0dcfc2aeae1bf3dac
paper_authors.name: Wen Zhang
paper_authors._id: 67e365b0dcfc2aeae1bf3dad
paper_authors.name: Huajun Chen
paper_title: ReSearch: Learning to Reason with Search for LLMs via Reinforcement
    Learning
paper_summary: Large Language Models (LLMs) have shown remarkable capabilities in reasoning,
  exemplified by the success of OpenAI-o1 and DeepSeek-R1. However, integrating
  reasoning with external search processes remains challenging, especially for
  complex multi-hop questions requiring multiple retrieval steps. We propose
  ReSearch, a novel framework that trains LLMs to Reason with Search via
  reinforcement learning without using any supervised data on reasoning steps.
  Our approach treats search operations as integral components of the reasoning
  chain, where when and how to perform searches is guided by text-based thinking,
  and search results subsequently influence further reasoning. We train ReSearch
  on Qwen2.5-7B(-Instruct) and Qwen2.5-32B(-Instruct) models and conduct
  extensive experiments. Despite being trained on only one dataset, our models
  demonstrate strong generalizability across various benchmarks. Analysis reveals
  that ReSearch naturally elicits advanced reasoning capabilities such as
  reflection and self-correction during the reinforcement learning process.
paper_ai_summary: ReSearch is a framework that trains LLMs to integrate reasoning with search using reinforcement learning, enhancing capabilities for complex multi-hop questions.
paper_ai_keywords: Large Language Models LLMs OpenAI-o1 DeepSeek-R1 ReSearch reinforcement learning Qwen2.5-7B Qwen2.5-32B