title: More Documents, Same Length: Isolating the Challenge of Multiple
    Documents in RAG
summary: Retrieval-augmented generation (RAG) provides LLMs with relevant documents.
  Although previous studies noted that retrieving many documents can degrade
  performance, they did not isolate how the quantity of documents affects
  performance while controlling for context length. We evaluate various language
  models on custom datasets derived from a multi-hop QA task. We keep the context
  length and position of relevant information constant while varying the number
  of documents, and find that increasing the document count in RAG settings poses
  significant challenges for LLMs. Additionally, our results indicate that
  processing multiple documents is a separate challenge from handling long
  contexts. We also make the datasets and code available:
  https://github.com/shaharl6000/MoreDocsSameLen .
paper_authors._id: 67d05aa2348bae81a8ae572e
paper_authors.name: Shahar Levy
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-14T08:58:54.727Z
paper_authors._id: 666d93d8357aeb334f1e9a7e
paper_authors.avatarUrl: /avatars/c240d1fc938271ce1d4321f5f3fb79d1.svg
paper_authors.fullname: Shahar Levy
paper_authors.type: user
paper_authors.user: Shahar6000
paper_authors._id: 67d05aa2348bae81a8ae572f
paper_authors.name: Nir Mazor
paper_authors._id: 67d05aa2348bae81a8ae5730
paper_authors.name: Lihi Shalmon
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-11T16:09:23.471Z
paper_authors._id: 63b433ee7af2e415f25b1a7b
paper_authors.avatarUrl: /avatars/0b03f66d263bffd22ed864d1241fe28b.svg
paper_authors.fullname: Lihi Shalmon
paper_authors.type: user
paper_authors.user: LihiShalmon
paper_authors._id: 67d05aa2348bae81a8ae5731
paper_authors.name: Michael Hassid
paper_authors._id: 67d05aa2348bae81a8ae5732
paper_authors.name: Gabriel Stanovsky
paper_title: More Documents, Same Length: Isolating the Challenge of Multiple
    Documents in RAG
paper_summary: Retrieval-augmented generation (RAG) provides LLMs with relevant documents.
  Although previous studies noted that retrieving many documents can degrade
  performance, they did not isolate how the quantity of documents affects
  performance while controlling for context length. We evaluate various language
  models on custom datasets derived from a multi-hop QA task. We keep the context
  length and position of relevant information constant while varying the number
  of documents, and find that increasing the document count in RAG settings poses
  significant challenges for LLMs. Additionally, our results indicate that
  processing multiple documents is a separate challenge from handling long
  contexts. We also make the datasets and code available:
  https://github.com/shaharl6000/MoreDocsSameLen .
paper_ai_keywords: Retrieval-augmented generation (RAG) LLMs relevant documents multi-hop QA task document count long contexts