Fine-tuning of downstream tasks It

is equally im-
portant to adapt Embedding models to downstream tasks.
When using RAG in downstream tasks, some works have
fine-tuned Embedding models by using the capabilities
of LLMs.PROMPTAGATOR[Dai et al., 2022] utilizes the
Large Language Model (LLM) as a few-shot query gener-
ator and creates task-specific retrievers based on the gen-
erated data, and alleviates the problem of supervised fine-
tuning, which is difficult
in some domains due to data
scarcity.LLM-Embedder[Zhang et al., 2023a]uses the Large
Language Model to output reward values for data from mul-
tiple downstream tasks, fine-tuning the retriever with two dif-
ferent supervised signals via hard labeling of the dataset and
the soft reward derived from LLM.

This somewhat
Retrieval-AugmentedGenerationforLargeLanguageModels-ASurvey