title: Chain-of-Retrieval Augmented Generation
summary: This paper introduces an approach for training o1-like RAG models that
  retrieve and reason over relevant information step by step before generating
  the final answer. Conventional RAG methods usually perform a single retrieval
  step before the generation process, which limits their effectiveness in
  addressing complex queries due to imperfect retrieval results. In contrast, our
  proposed method, CoRAG (Chain-of-Retrieval Augmented Generation), allows the
  model to dynamically reformulate the query based on the evolving state. To
  train CoRAG effectively, we utilize rejection sampling to automatically
  generate intermediate retrieval chains, thereby augmenting existing RAG
  datasets that only provide the correct final answer. At test time, we propose
  various decoding strategies to scale the model's test-time compute by
  controlling the length and number of sampled retrieval chains. Experimental
  results across multiple benchmarks validate the efficacy of CoRAG, particularly
  in multi-hop question answering tasks, where we observe more than 10 points
  improvement in EM score compared to strong baselines. On the KILT benchmark,
  CoRAG establishes a new state-of-the-art performance across a diverse range of
  knowledge-intensive tasks. Furthermore, we offer comprehensive analyses to
  understand the scaling behavior of CoRAG, laying the groundwork for future
  research aimed at developing factual and grounded foundation models.
paper_authors._id: 6796fa2d4fccd4b951e8ab0a
paper_authors.name: Liang Wang
paper_authors._id: 6796fa2d4fccd4b951e8ab0b
paper_authors.name: Haonan Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-13T08:26:16.425Z
paper_authors._id: 66add675c7a575aa0e03d5f3
paper_authors.avatarUrl: /avatars/b72b18130664c1de197c1f8df371aa70.svg
paper_authors.fullname: Haonan Chen
paper_authors.type: user
paper_authors.user: Haon-Chen
paper_authors._id: 6796fa2d4fccd4b951e8ab0c
paper_authors.name: Nan Yang
paper_authors._id: 6796fa2d4fccd4b951e8ab0d
paper_authors.name: Xiaolong Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-27T14:59:51.272Z
paper_authors._id: 64149ba8e682c73521d46ecb
paper_authors.avatarUrl: /avatars/c1859aa2d295077e798c8bdc83664a07.svg
paper_authors.fullname: Xiaolong Huang
paper_authors.type: user
paper_authors.user: DiamondH
paper_authors._id: 6796fa2d4fccd4b951e8ab0e
paper_authors.name: Zhicheng Dou
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-27T14:59:36.119Z
paper_authors._id: 66f0bf59e9d50ec57febf751
paper_authors.avatarUrl: /avatars/be97941e60064e5dd806c6fe9db3c537.svg
paper_authors.fullname: Zhicheng Dou
paper_authors.type: user
paper_authors.user: douzc
paper_authors._id: 6796fa2d4fccd4b951e8ab0f
paper_authors.name: Furu Wei
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-27T14:59:30.095Z
paper_authors._id: 6368c512fbfe97c16a40baba
paper_authors.avatarUrl: /avatars/1c23bc7c0b6d9225699ce27647623d7a.svg
paper_authors.fullname: Furu Wei
paper_authors.type: user
paper_authors.user: thegenerality
paper_title: Chain-of-Retrieval Augmented Generation
paper_summary: This paper introduces an approach for training o1-like RAG models that
  retrieve and reason over relevant information step by step before generating
  the final answer. Conventional RAG methods usually perform a single retrieval
  step before the generation process, which limits their effectiveness in
  addressing complex queries due to imperfect retrieval results. In contrast, our
  proposed method, CoRAG (Chain-of-Retrieval Augmented Generation), allows the
  model to dynamically reformulate the query based on the evolving state. To
  train CoRAG effectively, we utilize rejection sampling to automatically
  generate intermediate retrieval chains, thereby augmenting existing RAG
  datasets that only provide the correct final answer. At test time, we propose
  various decoding strategies to scale the model's test-time compute by
  controlling the length and number of sampled retrieval chains. Experimental
  results across multiple benchmarks validate the efficacy of CoRAG, particularly
  in multi-hop question answering tasks, where we observe more than 10 points
  improvement in EM score compared to strong baselines. On the KILT benchmark,
  CoRAG establishes a new state-of-the-art performance across a diverse range of
  knowledge-intensive tasks. Furthermore, we offer comprehensive analyses to
  understand the scaling behavior of CoRAG, laying the groundwork for future
  research aimed at developing factual and grounded foundation models.
paper_ai_summary: CoRAG, a multi-step retrieval and reasoning approach, enhances RAG models by dynamically refining queries and using rejection sampling to improve performance, especially in multi-hop question answering.
paper_ai_keywords: RAG models CoRAG rejection sampling intermediate retrieval chains decoding strategies EM score KILT benchmark multi-hop question answering