9.2 Hallucination-related Metrics in GQA
Currently, there is no automatic metric to evaluate hallucination in QGA specifically. While most
works on GQA use automatic evaluation metrics such as ROUGE score and F1 to measure the
quality of the answer, these N-gram overlap-based metrics are not a meaningful way to evaluate
hallucination due to their poor correlation with human judgments, as indicated by Krishna et al.
[88]. On the other hand, almost all the GQA-related work involves a human evaluation process as a
complement to the automatic evaluation. Normally, human annotators will be asked to assign a
score indicating the faithfulness of the answer, which can also be viewed as a measurement of the
answer hallucination. However, the metrics obtained via human evaluation come normally from a
small sample of the data.
SurveyofHallucinationinNatural Language Generation