At the chunk level, RETRO uses the previous chunk to re-
trieve the nearest neighboring chunk and integrates this infor-
mation with the contextual information of the previous chunk
to guide the generation of the next chunk. RETRO achieves
this by retrieving the nearest neighboring block N (Ci−1)
from the retrieval database, then fusing the contextual in-

by

for

embeds

relevant

formation of the preceding blocks (C1, . . . , Ci−1) and the
retrieval information of N (Ci−1) through cross-attention to
guide the generation of the next block Ci. To maintain causal-
ity, the autoregressive generation of the i-th block Ci can only
use the nearest neighbor of the previous block N (Ci−1) and
not N (Ci).
Augmented with Structured Data
Structured data sources like Knowledge Graphs (KG) are
gradually integrated into the paradigm of RAG. Verified KGs
can offer higher-quality context, reducing the likelihood of
model hallucinations.

knowledge

graph memory
from past dialogues
Retrieval-AugmentedGenerationforLargeLanguageModels-ASurvey