title: Scaling External Knowledge Input Beyond Context Windows of LLMs via
    Multi-Agent Collaboration
summary: With the rapid advancement of post-training techniques for reasoning and
  information seeking, large language models (LLMs) can incorporate a large
  quantity of retrieved knowledge to solve complex tasks. However, the limited
  context window of LLMs obstructs scaling the amount of external knowledge
  input, prohibiting further improvement, especially for tasks requiring
  significant amount of external knowledge. Existing context window extension
  methods inevitably cause information loss. LLM-based multi-agent methods emerge
  as a new paradigm to handle massive input in a distributional manner, where we
  identify two core bottlenecks in existing knowledge synchronization and
  reasoning processes. In this work, we develop a multi-agent framework,
  ExtAgents, to overcome the bottlenecks and enable better scalability
  in inference-time knowledge integration without longer-context training.
  Benchmarked with our enhanced multi-hop question answering test,
  $boldsymbol{inftyBench+}, and other public test sets including
  long survey generation, ExtAgents significantly enhances the performance over
  existing non-training methods with the same amount of external knowledge input,
  regardless of whether it falls within or exceeds the context window$.
  Moreover, the method maintains high efficiency due to high parallelism. Further
  study in the coordination of LLM agents on increasing external knowledge input
  could benefit real-world applications.
paper_authors._id: 6836ee74ce6ceb4c0d4ad9b9
paper_authors.name: Zijun Liu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-29T07:43:47.325Z
paper_authors._id: 6468c76bff18750165a64df3
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6468c76bff18750165a64df3/dHhE62SHOSJZjyU60vgh7.jpeg
paper_authors.fullname: Zijun Liu
paper_authors.type: user
paper_authors.user: BBQGOD
paper_authors._id: 6836ee74ce6ceb4c0d4ad9ba
paper_authors.name: Zhennan Wan
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-28T12:12:21.052Z
paper_authors._id: 65d73ac991dbd0f8b399ab73
paper_authors.avatarUrl: /avatars/c73ab59bd20a8de796e4f58a4147c4c9.svg
paper_authors.fullname: Zhennan Wan
paper_authors.type: user
paper_authors.user: zhennan1
paper_authors._id: 6836ee74ce6ceb4c0d4ad9bb
paper_authors.name: Peng Li
paper_authors._id: 6836ee74ce6ceb4c0d4ad9bc
paper_authors.name: Ming Yan
paper_authors._id: 6836ee74ce6ceb4c0d4ad9bd
paper_authors.name: Ji Zhang
paper_authors._id: 6836ee74ce6ceb4c0d4ad9be
paper_authors.name: Fei Huang
paper_authors._id: 6836ee74ce6ceb4c0d4ad9bf
paper_authors.name: Yang Liu
paper_title: Scaling External Knowledge Input Beyond Context Windows of LLMs via
    Multi-Agent Collaboration
paper_summary: With the rapid advancement of post-training techniques for reasoning and
  information seeking, large language models (LLMs) can incorporate a large
  quantity of retrieved knowledge to solve complex tasks. However, the limited
  context window of LLMs obstructs scaling the amount of external knowledge
  input, prohibiting further improvement, especially for tasks requiring
  significant amount of external knowledge. Existing context window extension
  methods inevitably cause information loss. LLM-based multi-agent methods emerge
  as a new paradigm to handle massive input in a distributional manner, where we
  identify two core bottlenecks in existing knowledge synchronization and
  reasoning processes. In this work, we develop a multi-agent framework,
  ExtAgents, to overcome the bottlenecks and enable better scalability
  in inference-time knowledge integration without longer-context training.
  Benchmarked with our enhanced multi-hop question answering test,
  $boldsymbol{inftyBench+}, and other public test sets including
  long survey generation, ExtAgents significantly enhances the performance over
  existing non-training methods with the same amount of external knowledge input,
  regardless of whether it falls within or exceeds the context window$.
  Moreover, the method maintains high efficiency due to high parallelism. Further
  study in the coordination of LLM agents on increasing external knowledge input
  could benefit real-world applications.
paper_ai_summary: The ExtAgents multi-agent framework enhances the scalability of inference-time knowledge integration in large language models, improving performance without increasing the context window.
paper_ai_keywords: large language models post-training techniques reasoning information seeking context window multi-agent framework knowledge synchronization reasoning processes multi-hop question answering long survey generation parallelism