Fine-tuning generator can result

in outputs that are
more stylized and customized. On one hand,
it allows
for specialized adaptation to different input data formats.
For example, fine-tuning LLMs to fit
the structure of
knowledge graphs [Kang et al., 2023], the structure of text
pairs [Kang et al., 2023, Cheng et al., 2023b], and other spe-
cific structures [Li et al., 2023d]. On the other hand, by con-
structing directive datasets, one can demand LLMs to gen-
erate specific formats content. For instance, in adaptive or
iterative retrieval scenarios, LLMs are fine-tuned to generate
content that will help determine the timing for the next step
of action [Jiang et al., 2023b, Asai et al., 2023].

By synergistically fine-tuning both the retriever and the
generator, we can enhance the model’s generalization capa-
RAG forLargeLanguageModels-ASurvey