6 Augmentation in RAG
This section is structured around three key aspects: the aug-
mentation stage, sources of augmentation data, and the aug-
mentation process. These facets elucidate the critical tech-
nologies pivotal to RAG’s development. A taxonomy of
RAG’s core components is presented in Figure 4.
6.1 RAG in Augmentation Stages
RAG, a knowledge-intensive endeavor, incorporates a vari-
ety of technical methodologies across the pre-training, fine-
tuning, and inference stages of language model training.
Pre-training Stage
During the pre-training stage, researchers have investigated
methods to bolster PTMs for open-domain QA through

Figure 4: Taxonomy of RAG’s core components

retrieval-based strategies. The REALM model adopts a struc-
tured, interpretable method for knowledge embedding, fram-
ing pre-training, and fine-tuning as a retrieve-then-predict
workflow within the masked language model (MLM) frame-
work [Arora et al., 2023] .
RAG forLargeLanguageModels-ASurvey