Historically, RAG models assessments have centered
on their execution in specific downstream tasks. These
evaluations employ established metrics suitable to the tasks
at hand.
For instance, question answering evaluations
rely on EM and F1 scores [Wang et al., 2023a,
might
Shi et al., 2023, Feng et al., 2023, Ma et al., 2023a], whereas
fact-checking tasks often hinge on accuracy as the pri-
Izacard et al., 2022,
mary metric
Shao et al., 2023]. Tools like RALLE, designed for the auto-
matic evaluation of RAG applications, similarly base their as-
sessments on these task-specific metrics [Hoshi et al., 2023].
Despite this, there is a notable paucity of research dedicated
to evaluating the distinct characteristics of RAG models, with
only a handful of related studies.

[Lewis et al., 2020,
RAG forLargeLanguageModels-ASurvey