Domain Knowledge Fine-tuning. To ensure that an embed-
ding model accurately captures domain-specific information,
it is imperative to utilize domain-specific datasets for fine-
tuning. This process diverges from standard language model
fine-tuning, chiefly in the nature of the datasets involved.
Typically, the dataset for embedding model fine-tuning en-
compasses three principal elements: queries, a corpus, and
relevant documents. The model employs these queries to
identify pertinent documents within the corpus. The effi-
cacy of the model is then gauged based on its ability to re-
trieve these relevant documents in response to the queries.
The dataset construction, model fine-tuning, and evalua-
tion phases each present distinct challenges. The LlamaIn-
dex [Liu, 2023] introduces a suite of pivotal classes and func-
tions designed to enhance the embedding model fine-tuning
workflow, thereby simplifying these intricate processes. By
RAG forLargeLanguageModels-ASurvey