NLI-based Metrics. As mentioned in Section 4, the NLI-model (textual entailment model) can
be utilized to measure hallucination based on the assumption that a faithful summary will be
entailed by the gold source. However, Falke et al. [45] discover that models trained on NLI datasets
can not transfer well to abstractive summarization tasks, degrading the reliability of NLI-based
hallucination metrics. To improve NLI models for hallucination evaluation, they release collected
annotations as additional test data. Other efforts have also been made to further improve NLI
models. Mishra et al. [131] find that the low performance of NLI-based metrics is mainly caused by
the length of the premises in NLI datasets being shorter than the source documents in abstractive
summarization. Thus, the authors propose to convert multiple-choice reading comprehension
datasets into long premise NLI datasets automatically. The results indicate that long-premise NLI
SurveyofHallucinationinNatural Language Generation