Purely parameterized language models (LLMs) store their
world knowledge, which is acquired from vast corpora, in
the parameters of the model. Nevertheless, such models have
their limitations. Firstly, it is difficult to retain all the knowl-
edge from the training corpus, especially for less common
and more specific knowledge. Secondly, since the model
parameters cannot be updated dynamically, the parametric
knowledge is susceptible to becoming outdated over time.
Lastly, an expansion in parameters leads to increased com-

putational expenses for both training and inference. To ad-
dress the limitations of purely parameterized models, lan-
guage models can adopt a semi-parameterized approach by
integrating a non-parameterized corpus database with pa-
rameterized models. This approach is known as Retrieval-
Augmented Generation (RAG).
Retrieval-AugmentedGenerationforLargeLanguageModels-ASurvey