title: LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMs
summary: In traditional RAG framework, the basic retrieval units are normally short.
  The common retrievers like DPR normally work with 100-word Wikipedia
  paragraphs. Such a design forces the retriever to search over a large corpus to
  find the `needle' unit. In contrast, the readers only need to extract answers
  from the short retrieved units. Such an imbalanced `heavy' retriever and
  `light' reader design can lead to sub-optimal performance. In order to
  alleviate the imbalance, we propose a new framework LongRAG, consisting of a
  `long retriever' and a `long reader'. LongRAG processes the entire Wikipedia
  into 4K-token units, which is 30x longer than before. By increasing the unit
  size, we significantly reduce the total units from 22M to 700K. This
  significantly lowers the burden of retriever, which leads to a remarkable
  retrieval score: answer recall@1=71% on NQ (previously 52%) and answer
  recall@2=72% (previously 47%) on HotpotQA (full-wiki). Then we feed the top-k
  retrieved units (approx 30K tokens) to an existing long-context LLM to
  perform zero-shot answer extraction. Without requiring any training, LongRAG
  achieves an EM of 62.7% on NQ, which is the best known result. LongRAG also
  achieves 64.3% on HotpotQA (full-wiki), which is on par of the SoTA model. Our
  study offers insights into the future roadmap for combining RAG with
  long-context LLMs.
paper_authors._id: 6678e77c66539bc0a08db180
paper_authors.name: Ziyan Jiang
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-06-24T03:32:43.711Z
paper_authors._id: 64778fb8168cb428e00f69b0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64778fb8168cb428e00f69b0/D_XYg74zHG9K3HUJj_gD4.jpeg
paper_authors.fullname: Ziyan Jiang
paper_authors.type: user
paper_authors.user: ziyjiang
paper_authors._id: 6678e77c66539bc0a08db181
paper_authors.name: Xueguang Ma
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-06-26T18:51:44.209Z
paper_authors._id: 5ec82854968f6028e0559f70
paper_authors.avatarUrl: /avatars/45b58d912f7d00cb351947cd79d5eeb4.svg
paper_authors.fullname: Xueguang Ma
paper_authors.type: user
paper_authors.user: MrLight
paper_authors._id: 6678e77c66539bc0a08db182
paper_authors.name: Wenhu Chen
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-06-24T03:26:53.799Z
paper_authors._id: 6313a86154e6e5d9f0f94e04
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1662232951344-6313a86154e6e5d9f0f94e04.jpeg
paper_authors.fullname: Wenhu Chen
paper_authors.type: user
paper_authors.user: wenhu
paper_title: LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMs
paper_summary: In traditional RAG framework, the basic retrieval units are normally short.
  The common retrievers like DPR normally work with 100-word Wikipedia
  paragraphs. Such a design forces the retriever to search over a large corpus to
  find the `needle' unit. In contrast, the readers only need to extract answers
  from the short retrieved units. Such an imbalanced `heavy' retriever and
  `light' reader design can lead to sub-optimal performance. In order to
  alleviate the imbalance, we propose a new framework LongRAG, consisting of a
  `long retriever' and a `long reader'. LongRAG processes the entire Wikipedia
  into 4K-token units, which is 30x longer than before. By increasing the unit
  size, we significantly reduce the total units from 22M to 700K. This
  significantly lowers the burden of retriever, which leads to a remarkable
  retrieval score: answer recall@1=71% on NQ (previously 52%) and answer
  recall@2=72% (previously 47%) on HotpotQA (full-wiki). Then we feed the top-k
  retrieved units (approx 30K tokens) to an existing long-context LLM to
  perform zero-shot answer extraction. Without requiring any training, LongRAG
  achieves an EM of 62.7% on NQ, which is the best known result. LongRAG also
  achieves 64.3% on HotpotQA (full-wiki), which is on par of the SoTA model. Our
  study offers insights into the future roadmap for combining RAG with
  long-context LLMs.
paper_ai_summary: The LongRAG framework improves information retrieval and answer extraction by using longer retrieval units and leveraging long-context LLMs, leading to better performance on benchmark datasets.
paper_ai_keywords: LongRAG retriever reader 4K-token units long-context LLMs answer recall@1 answer recall@2 EM NQ HotpotQA full-wiki