Variants of F1 Metrics. Knowledge F1 (KF1) measures the overlap between the generated re-
sponses and the gold knowledge sentences to which the human referred for conversation during
dataset collection [168]. KF1 attempts to capture whether a model can generate knowledgable
responses by correctly utilizing the relevant knowledge. KF1 is only available for datasets with
labeled ground-truth knowledge. Shuster et al. [168] further propose Rare F1 (RF1), which only
considers the infrequent words in the dataset when calculating F1 to avoid influence from the
common uni-grams. The authors define an infrequent word as being in the lower half of the
cumulative frequency distribution of the reference corpus.
SurveyofHallucinationinNatural Language Generation