Generalization. Although we can see that the source and output text of different tasks are in
various forms, investigating their relationship and common ground and proposing general metrics
to evaluate hallucinations are worth exploring. Task-agnostic metrics with cross-domain robustness
could help the research community to build a unified benchmark. It is also important and meaningful
to build open-source platforms to collaborate and standardize the evaluation metrics for NLG tasks.
Incorporation of Human Cognitive Perspective. A good automatic metric should correlate with
human evaluation. Humans are sensitive to different types of information. For instance, proper
nouns are usually more important than pronouns in the generated text. Mistakes concerning named
entities are striking to human users, but automatic metrics treat them equally if not properly
designed. In order to address this issue, new metrics should be designed from the human cognitive
SurveyofHallucinationinNatural Language Generation