Training Language Models on the Knowledge Graph: Insights on
  Hallucinations and Their Detectability
While many capabilities of language models (LMs) improve with increased
training budget, the influence of scale on hallucinations is not yet fully
understood. Hallucinations come in many forms, and there is no universally
accepted definition. We thus focus on studying only those hallucinations where
a correct answer appears verbatim in the training set. To fully control the
training data content, we construct a knowledge graph (KG)-based dataset, and
use it to train a set of increasingly large LMs. We find that for a fixed
dataset, larger and longer-trained LMs hallucinate less. However, hallucinating
on leq5% of the training data requires an order of magnitude larger model,
and thus an order of magnitude more compute, than Hoffmann et al. (2022)
reported was optimal. Given this costliness, we study how hallucination
detectors depend on scale. While we see detector size improves performance on
fixed LM's outputs, we find an inverse relationship between the scale of the LM
and the detectability of its hallucinations.
66bed14c3aa0cce225da0cde
Jiri Hron
66bed14c3aa0cce225da0cdf
Laura Culp
66bed14c3aa0cce225da0ce0
Gamaleldin Elsayed
66bed14c3aa0cce225da0ce1
Rosanne Liu
66bed14c3aa0cce225da0ce2
Ben Adlam
66bed14c3aa0cce225da0ce3
Maxwell Bileschi
66bed14c3aa0cce225da0ce4
Bernd Bohnet
66bed14c3aa0cce225da0ce5
JD Co-Reyes
66bed14c3aa0cce225da0ce6
Noah Fiedel
66bed14c3aa0cce225da0ce7
C. Daniel Freeman
66bed14c3aa0cce225da0ce8
Izzeddin Gur
66bed14c3aa0cce225da0ce9
Kathleen Kenealy
66bed14c3aa0cce225da0cea
Jaehoon Lee
66bed14c3aa0cce225da0ceb
Peter J. Liu
66bed14c3aa0cce225da0cec
Gaurav Mishra
66bed14c3aa0cce225da0ced
Igor Mordatch
66bed14c3aa0cce225da0cee
Azade Nova
66bed14c3aa0cce225da0cef
Roman Novak
66bed14c3aa0cce225da0cf0
Aaron Parisi
66bed14c3aa0cce225da0cf1
Jeffrey Pennington
66bed14c3aa0cce225da0cf2
Alex Rizkowsky
66bed14c3aa0cce225da0cf3
Isabelle Simpson
66bed14c3aa0cce225da0cf4
Hanie Sedghi
66bed14c3aa0cce225da0cf5
Jascha Sohl-dickstein
66bed14c3aa0cce225da0cf6
Kevin Swersky
66bed14c3aa0cce225da0cf7
Sharad Vikram
66bed14c3aa0cce225da0cf8
Tris Warkentin
66bed14c3aa0cce225da0cf9
Lechao Xiao
66bed14c3aa0cce225da0cfa
Kelvin Xu
66bed14c3aa0cce225da0cfb
Jasper Snoek
66bed14c3aa0cce225da0cfc
Simon Kornblith
Training Language Models on the Knowledge Graph: Insights on
  Hallucinations and Their Detectability
While many capabilities of language models (LMs) improve with increased
training budget, the influence of scale on hallucinations is not yet fully
understood. Hallucinations come in many forms, and there is no universally
accepted definition. We thus focus on studying only those hallucinations where
a correct answer appears verbatim in the training set. To fully control the
training data content, we construct a knowledge graph (KG)-based dataset, and
use it to train a set of increasingly large LMs. We find that for a fixed
dataset, larger and longer-trained LMs hallucinate less. However, hallucinating
on leq5% of the training data requires an order of magnitude larger model,
and thus an order of magnitude more compute, than Hoffmann et al. (2022)
reported was optimal. Given this costliness, we study how hallucination
detectors depend on scale. While we see detector size improves performance on
fixed LM's outputs, we find an inverse relationship between the scale of the LM
and the detectability of its hallucinations.
Larger language models trained on fixed datasets exhibit fewer hallucinations, but detecting these hallucinations becomes more difficult as model size increases, despite improvements in detector size.
knowledge graph-based dataset language models hallucinations detector size detectability