title: Hallucination Detox: Sensitive Neuron Dropout (SeND) for Large Language
    Model Training
summary: As large language models (LLMs) become increasingly deployed across various
  industries, concerns regarding their reliability, particularly due to
  hallucinations-outputs that are factually inaccurate or irrelevant to user
  input-have grown. Our research investigates the relationship between the
  training process and the emergence of hallucinations to address a key gap in
  existing research that focuses primarily on post hoc detection and mitigation
  strategies. Using models from the Pythia suite (70M-12B parameters) and several
  hallucination detection metrics, we analyze hallucination trends throughout
  training and explore LLM internal dynamics. We introduce SEnsitive Neuron
  Dropout (SeND), a novel training protocol designed to mitigate hallucinations
  by reducing variance during training. SeND achieves this by deterministically
  dropping neurons with significant variability on a dataset, referred to as
  Sensitive Neurons. In addition, we develop an unsupervised hallucination
  detection metric, Efficient EigenScore (EES), which approximates the
  traditional EigenScore in 2x speed. This efficient metric is integrated into
  our protocol, allowing SeND to be both computationally scalable and effective
  at reducing hallucinations. Our empirical evaluation demonstrates that our
  approach improves LLM reliability at test time by up to 40% compared to normal
  training while also providing an efficient method to improve factual accuracy
  when adapting LLMs to domains such as Wikipedia and Medical datasets.
paper_authors._id: 6717a5dadcef4aa5a830ef29
paper_authors.name: Shahrad Mohammadzadeh
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-22T13:28:05.645Z
paper_authors._id: 659d84dba74945e564da065b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/659d84dba74945e564da065b/NMavmXi1gkmtKO0GI-HKX.png
paper_authors.fullname: MZ
paper_authors.type: user
paper_authors.user: Shahradmz
paper_authors._id: 6717a5dadcef4aa5a830ef2a
paper_authors.name: Juan David Guerra
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-30T09:55:21.596Z
paper_authors._id: 66c4f7fb1b183af250c17b08
paper_authors.avatarUrl: /avatars/8ef4759bfc2c1031100bb430270c18ac.svg
paper_authors.fullname: Juan David Guerra
paper_authors.type: user
paper_authors.user: JuanDG
paper_authors._id: 6717a5dadcef4aa5a830ef2b
paper_authors.name: Marco Bonizzato
paper_authors._id: 6717a5dadcef4aa5a830ef2c
paper_authors.name: Reihaneh Rabbany
paper_authors._id: 6717a5dadcef4aa5a830ef2d
paper_authors.name: Golnoosh Farnadi
paper_title: Hallucination Detox: Sensitive Neuron Dropout (SeND) for Large Language
    Model Training
paper_summary: As large language models (LLMs) become increasingly deployed across various
  industries, concerns regarding their reliability, particularly due to
  hallucinations-outputs that are factually inaccurate or irrelevant to user
  input-have grown. Our research investigates the relationship between the
  training process and the emergence of hallucinations to address a key gap in
  existing research that focuses primarily on post hoc detection and mitigation
  strategies. Using models from the Pythia suite (70M-12B parameters) and several
  hallucination detection metrics, we analyze hallucination trends throughout
  training and explore LLM internal dynamics. We introduce SEnsitive Neuron
  Dropout (SeND), a novel training protocol designed to mitigate hallucinations
  by reducing variance during training. SeND achieves this by deterministically
  dropping neurons with significant variability on a dataset, referred to as
  Sensitive Neurons. In addition, we develop an unsupervised hallucination
  detection metric, Efficient EigenScore (EES), which approximates the
  traditional EigenScore in 2x speed. This efficient metric is integrated into
  our protocol, allowing SeND to be both computationally scalable and effective
  at reducing hallucinations. Our empirical evaluation demonstrates that our
  approach improves LLM reliability at test time by up to 40% compared to normal
  training while also providing an efficient method to improve factual accuracy
  when adapting LLMs to domains such as Wikipedia and Medical datasets.
paper_ai_keywords: large language models (LLMs) hallucinations factually inaccurate irrelevant training process Pythia suite hallucination detection compression iterative compression training dynamics Sensitive Neuron Dropout (SeND) neuron variability Sensitive Neurons unsupervised hallucination detection Efficient EigenScore (EES) EigenScore computational scalability LLM reliability factual accuracy Wikipedia dataset Medical datasets