valid knowledge triple in the gold reference knowledge. Recently, there have been some attempts to
generate informative responses without explicit knowledge inputs, but with the help of the implicit
knowledge inside large pre-trained language models instead [217, 239] during the inference time.
Under this setting, the study of extrinsic hallucination is of great value but still poorly investigated.
8.2.3 Hallucination Metrics. For generation-based dialogue systems, especially open-domain chat-
bots, the hallucination evaluation method remains an open problem [160]. As of now, there is
no standard metric. Therefore, chatbots are usually evaluated by humans on factual consistency
or factual correctness [163, 210]. We also introduce some automatic statistical and model-based
metrics as a reference, which will be described in more detail below.
SurveyofHallucinationinNatural Language Generation