Scaling External Knowledge Input Beyond Context Windows of LLMs via
  Multi-Agent Collaboration
With the rapid advancement of post-training techniques for reasoning and
information seeking, large language models (LLMs) can incorporate a large
quantity of retrieved knowledge to solve complex tasks. However, the limited
context window of LLMs obstructs scaling the amount of external knowledge
input, prohibiting further improvement, especially for tasks requiring
significant amount of external knowledge. Existing context window extension
methods inevitably cause information loss. LLM-based multi-agent methods emerge
as a new paradigm to handle massive input in a distributional manner, where we
identify two core bottlenecks in existing knowledge synchronization and
reasoning processes. In this work, we develop a multi-agent framework,
ExtAgents, to overcome the bottlenecks and enable better scalability
in inference-time knowledge integration without longer-context training.
Benchmarked with our enhanced multi-hop question answering test,
$boldsymbol{inftyBench+}, and other public test sets including
long survey generation, ExtAgents significantly enhances the performance over
existing non-training methods with the same amount of external knowledge input,
regardless of whether it falls within or exceeds the context window$.
Moreover, the method maintains high efficiency due to high parallelism. Further
study in the coordination of LLM agents on increasing external knowledge input
could benefit real-world applications.
6836ee74ce6ceb4c0d4ad9b9
Zijun Liu
claimed_verified
2025-05-29T07:43:47.325Z
6468c76bff18750165a64df3
https://cdn-avatars.huggingface.co/v1/production/uploads/6468c76bff18750165a64df3/dHhE62SHOSJZjyU60vgh7.jpeg
Zijun Liu
user
BBQGOD
6836ee74ce6ceb4c0d4ad9ba
Zhennan Wan
claimed_verified
2025-05-28T12:12:21.052Z
65d73ac991dbd0f8b399ab73
/avatars/c73ab59bd20a8de796e4f58a4147c4c9.svg
Zhennan Wan
user
zhennan1
6836ee74ce6ceb4c0d4ad9bb
Peng Li
6836ee74ce6ceb4c0d4ad9bc
Ming Yan
6836ee74ce6ceb4c0d4ad9bd
Ji Zhang
6836ee74ce6ceb4c0d4ad9be
Fei Huang
6836ee74ce6ceb4c0d4ad9bf
Yang Liu
Scaling External Knowledge Input Beyond Context Windows of LLMs via
  Multi-Agent Collaboration
With the rapid advancement of post-training techniques for reasoning and
information seeking, large language models (LLMs) can incorporate a large
quantity of retrieved knowledge to solve complex tasks. However, the limited
context window of LLMs obstructs scaling the amount of external knowledge
input, prohibiting further improvement, especially for tasks requiring
significant amount of external knowledge. Existing context window extension
methods inevitably cause information loss. LLM-based multi-agent methods emerge
as a new paradigm to handle massive input in a distributional manner, where we
identify two core bottlenecks in existing knowledge synchronization and
reasoning processes. In this work, we develop a multi-agent framework,
ExtAgents, to overcome the bottlenecks and enable better scalability
in inference-time knowledge integration without longer-context training.
Benchmarked with our enhanced multi-hop question answering test,
$boldsymbol{inftyBench+}, and other public test sets including
long survey generation, ExtAgents significantly enhances the performance over
existing non-training methods with the same amount of external knowledge input,
regardless of whether it falls within or exceeds the context window$.
Moreover, the method maintains high efficiency due to high parallelism. Further
study in the coordination of LLM agents on increasing external knowledge input
could benefit real-world applications.
The ExtAgents multi-agent framework enhances the scalability of inference-time knowledge integration in large language models, improving performance without increasing the context window.
large language models post-training techniques reasoning information seeking context window multi-agent framework knowledge synchronization reasoning processes multi-hop question answering long survey generation parallelism