title: Training Language Models on the Knowledge Graph: Insights on
    Hallucinations and Their Detectability
summary: While many capabilities of language models (LMs) improve with increased
  training budget, the influence of scale on hallucinations is not yet fully
  understood. Hallucinations come in many forms, and there is no universally
  accepted definition. We thus focus on studying only those hallucinations where
  a correct answer appears verbatim in the training set. To fully control the
  training data content, we construct a knowledge graph (KG)-based dataset, and
  use it to train a set of increasingly large LMs. We find that for a fixed
  dataset, larger and longer-trained LMs hallucinate less. However, hallucinating
  on leq5% of the training data requires an order of magnitude larger model,
  and thus an order of magnitude more compute, than Hoffmann et al. (2022)
  reported was optimal. Given this costliness, we study how hallucination
  detectors depend on scale. While we see detector size improves performance on
  fixed LM's outputs, we find an inverse relationship between the scale of the LM
  and the detectability of its hallucinations.
paper_authors._id: 66bed14c3aa0cce225da0cde
paper_authors.name: Jiri Hron
paper_authors._id: 66bed14c3aa0cce225da0cdf
paper_authors.name: Laura Culp
paper_authors._id: 66bed14c3aa0cce225da0ce0
paper_authors.name: Gamaleldin Elsayed
paper_authors._id: 66bed14c3aa0cce225da0ce1
paper_authors.name: Rosanne Liu
paper_authors._id: 66bed14c3aa0cce225da0ce2
paper_authors.name: Ben Adlam
paper_authors._id: 66bed14c3aa0cce225da0ce3
paper_authors.name: Maxwell Bileschi
paper_authors._id: 66bed14c3aa0cce225da0ce4
paper_authors.name: Bernd Bohnet
paper_authors._id: 66bed14c3aa0cce225da0ce5
paper_authors.name: JD Co-Reyes
paper_authors._id: 66bed14c3aa0cce225da0ce6
paper_authors.name: Noah Fiedel
paper_authors._id: 66bed14c3aa0cce225da0ce7
paper_authors.name: C. Daniel Freeman
paper_authors._id: 66bed14c3aa0cce225da0ce8
paper_authors.name: Izzeddin Gur
paper_authors._id: 66bed14c3aa0cce225da0ce9
paper_authors.name: Kathleen Kenealy
paper_authors._id: 66bed14c3aa0cce225da0cea
paper_authors.name: Jaehoon Lee
paper_authors._id: 66bed14c3aa0cce225da0ceb
paper_authors.name: Peter J. Liu
paper_authors._id: 66bed14c3aa0cce225da0cec
paper_authors.name: Gaurav Mishra
paper_authors._id: 66bed14c3aa0cce225da0ced
paper_authors.name: Igor Mordatch
paper_authors._id: 66bed14c3aa0cce225da0cee
paper_authors.name: Azade Nova
paper_authors._id: 66bed14c3aa0cce225da0cef
paper_authors.name: Roman Novak
paper_authors._id: 66bed14c3aa0cce225da0cf0
paper_authors.name: Aaron Parisi
paper_authors._id: 66bed14c3aa0cce225da0cf1
paper_authors.name: Jeffrey Pennington
paper_authors._id: 66bed14c3aa0cce225da0cf2
paper_authors.name: Alex Rizkowsky
paper_authors._id: 66bed14c3aa0cce225da0cf3
paper_authors.name: Isabelle Simpson
paper_authors._id: 66bed14c3aa0cce225da0cf4
paper_authors.name: Hanie Sedghi
paper_authors._id: 66bed14c3aa0cce225da0cf5
paper_authors.name: Jascha Sohl-dickstein
paper_authors._id: 66bed14c3aa0cce225da0cf6
paper_authors.name: Kevin Swersky
paper_authors._id: 66bed14c3aa0cce225da0cf7
paper_authors.name: Sharad Vikram
paper_authors._id: 66bed14c3aa0cce225da0cf8
paper_authors.name: Tris Warkentin
paper_authors._id: 66bed14c3aa0cce225da0cf9
paper_authors.name: Lechao Xiao
paper_authors._id: 66bed14c3aa0cce225da0cfa
paper_authors.name: Kelvin Xu
paper_authors._id: 66bed14c3aa0cce225da0cfb
paper_authors.name: Jasper Snoek
paper_authors._id: 66bed14c3aa0cce225da0cfc
paper_authors.name: Simon Kornblith
paper_title: Training Language Models on the Knowledge Graph: Insights on
    Hallucinations and Their Detectability
paper_summary: While many capabilities of language models (LMs) improve with increased
  training budget, the influence of scale on hallucinations is not yet fully
  understood. Hallucinations come in many forms, and there is no universally
  accepted definition. We thus focus on studying only those hallucinations where
  a correct answer appears verbatim in the training set. To fully control the
  training data content, we construct a knowledge graph (KG)-based dataset, and
  use it to train a set of increasingly large LMs. We find that for a fixed
  dataset, larger and longer-trained LMs hallucinate less. However, hallucinating
  on leq5% of the training data requires an order of magnitude larger model,
  and thus an order of magnitude more compute, than Hoffmann et al. (2022)
  reported was optimal. Given this costliness, we study how hallucination
  detectors depend on scale. While we see detector size improves performance on
  fixed LM's outputs, we find an inverse relationship between the scale of the LM
  and the detectability of its hallucinations.
paper_ai_keywords: knowledge Graph (KG)-based dataset large LMs longer-trained LMs hallucination detectors