2.1.1.4 End-to-End RAG
The end-to-end process of RAG proposed in the
paper by (Lewis et al., 2021) involves integrating a
pre-trained sequence-to-sequence (seq2seq) trans-
former with a dense vector index of Wikipedia, ac-
cessed through the Dense Passage Retriever (DPR).
This innovative combination allows the model to
condition its output generation on both the input
query and latent documents provided by the DPR.
In this process, the DPR acts as a neural retriever,
supplying relevant documents based on the input.
These documents are then used by the seq2seq
model, specifically BART, to generate the final out-
put. The model employs a top-K approximation
to marginalize these latent documents, which can
be done on a per-output basis (assuming one doc-
ument is responsible for all tokens) or a per-token
basis (allowing different documents to influence
different parts of the output).
AComprehensiveSurveyofHallucinationMitigationTechniquesinLarge LanguageModels