RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval
Retrieval-augmented language models can better adapt to changes in world
state and incorporate long-tail knowledge. However, most existing methods
retrieve only short contiguous chunks from a retrieval corpus, limiting
holistic understanding of the overall document context. We introduce the novel
approach of recursively embedding, clustering, and summarizing chunks of text,
constructing a tree with differing levels of summarization from the bottom up.
At inference time, our RAPTOR model retrieves from this tree, integrating
information across lengthy documents at different levels of abstraction.
Controlled experiments show that retrieval with recursive summaries offers
significant improvements over traditional retrieval-augmented LMs on several
tasks. On question-answering tasks that involve complex, multi-step reasoning,
we show state-of-the-art results; for example, by coupling RAPTOR retrieval
with the use of GPT-4, we can improve the best performance on the QuALITY
benchmark by 20% in absolute accuracy.
65bb083bf8b6408e7f2feab4
Parth Sarthi
claimed_verified
2024-02-02T09:01:30.143Z
638d581a61eb5101751f0da7
/avatars/61d2a707ab9d27db9cd7725e0f937196.svg
Parth Sarthi
user
psarthi
65bb083bf8b6408e7f2feab5
Salman Abdullah
claimed_verified
2024-02-08T09:50:20.618Z
646a8ef48dfd6ff79b6c33d2
https://cdn-avatars.huggingface.co/v1/production/uploads/646a8ef48dfd6ff79b6c33d2/Od83-X9bIwDo5MEirKX3f.jpeg
Salman Abdullah
user
salman-abdullah
65bb083bf8b6408e7f2feab6
Aditi Tuli
65bb083bf8b6408e7f2feab7
Shubh Khanna
admin_assigned
2024-02-01T11:29:22.728Z
639481afacf644bed584e8de
/avatars/67be7c6058f3fa5106b8691b2554146b.svg
Shubh Khanna
user
skhanna
65bb083bf8b6408e7f2feab8
Anna Goldie
65bb083bf8b6408e7f2feab9
Christopher D. Manning
admin_assigned
2024-02-01T11:30:04.209Z
5f555a1f8bf55658acfed19e
https://cdn-avatars.huggingface.co/v1/production/uploads/1628695407761-5f555a1f8bf55658acfed19e.jpeg
Christopher Manning
user
manning
RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval
Retrieval-augmented language models can better adapt to changes in world
state and incorporate long-tail knowledge. However, most existing methods
retrieve only short contiguous chunks from a retrieval corpus, limiting
holistic understanding of the overall document context. We introduce the novel
approach of recursively embedding, clustering, and summarizing chunks of text,
constructing a tree with differing levels of summarization from the bottom up.
At inference time, our RAPTOR model retrieves from this tree, integrating
information across lengthy documents at different levels of abstraction.
Controlled experiments show that retrieval with recursive summaries offers
significant improvements over traditional retrieval-augmented LMs on several
tasks. On question-answering tasks that involve complex, multi-step reasoning,
we show state-of-the-art results; for example, by coupling RAPTOR retrieval
with the use of GPT-4, we can improve the best performance on the QuALITY
benchmark by 20% in absolute accuracy.
The RAPTOR model enhances retrieval-augmented language models by recursively embedding, clustering, and summarizing text chunks, leading to better performance on question-answering tasks involving complex reasoning.
retrieval-augmented language models Recursive embedding clustering summarizing tree structure abstraction levels RAPTOR model question-answering tasks multi-step reasoning QuALITY benchmark