Hallucination Detox: Sensitive Neuron Dropout (SeND) for Large Language
  Model Training
As large language models (LLMs) become increasingly deployed across various
industries, concerns regarding their reliability, particularly due to
hallucinations-outputs that are factually inaccurate or irrelevant to user
input-have grown. Our research investigates the relationship between the
training process and the emergence of hallucinations to address a key gap in
existing research that focuses primarily on post hoc detection and mitigation
strategies. Using models from the Pythia suite (70M-12B parameters) and several
hallucination detection metrics, we analyze hallucination trends throughout
training and explore LLM internal dynamics. We introduce SEnsitive Neuron
Dropout (SeND), a novel training protocol designed to mitigate hallucinations
by reducing variance during training. SeND achieves this by deterministically
dropping neurons with significant variability on a dataset, referred to as
Sensitive Neurons. In addition, we develop an unsupervised hallucination
detection metric, Efficient EigenScore (EES), which approximates the
traditional EigenScore in 2x speed. This efficient metric is integrated into
our protocol, allowing SeND to be both computationally scalable and effective
at reducing hallucinations. Our empirical evaluation demonstrates that our
approach improves LLM reliability at test time by up to 40% compared to normal
training while also providing an efficient method to improve factual accuracy
when adapting LLMs to domains such as Wikipedia and Medical datasets.
6717a5dadcef4aa5a830ef29
Shahrad Mohammadzadeh
claimed_verified
2024-10-22T13:28:05.645Z
659d84dba74945e564da065b
https://cdn-avatars.huggingface.co/v1/production/uploads/659d84dba74945e564da065b/NMavmXi1gkmtKO0GI-HKX.png
MZ
user
Shahradmz
6717a5dadcef4aa5a830ef2a
Juan David Guerra
claimed_verified
2024-10-30T09:55:21.596Z
66c4f7fb1b183af250c17b08
/avatars/8ef4759bfc2c1031100bb430270c18ac.svg
Juan David Guerra
user
JuanDG
6717a5dadcef4aa5a830ef2b
Marco Bonizzato
6717a5dadcef4aa5a830ef2c
Reihaneh Rabbany
6717a5dadcef4aa5a830ef2d
Golnoosh Farnadi
Hallucination Detox: Sensitive Neuron Dropout (SeND) for Large Language
  Model Training
As large language models (LLMs) become increasingly deployed across various
industries, concerns regarding their reliability, particularly due to
hallucinations-outputs that are factually inaccurate or irrelevant to user
input-have grown. Our research investigates the relationship between the
training process and the emergence of hallucinations to address a key gap in
existing research that focuses primarily on post hoc detection and mitigation
strategies. Using models from the Pythia suite (70M-12B parameters) and several
hallucination detection metrics, we analyze hallucination trends throughout
training and explore LLM internal dynamics. We introduce SEnsitive Neuron
Dropout (SeND), a novel training protocol designed to mitigate hallucinations
by reducing variance during training. SeND achieves this by deterministically
dropping neurons with significant variability on a dataset, referred to as
Sensitive Neurons. In addition, we develop an unsupervised hallucination
detection metric, Efficient EigenScore (EES), which approximates the
traditional EigenScore in 2x speed. This efficient metric is integrated into
our protocol, allowing SeND to be both computationally scalable and effective
at reducing hallucinations. Our empirical evaluation demonstrates that our
approach improves LLM reliability at test time by up to 40% compared to normal
training while also providing an efficient method to improve factual accuracy
when adapting LLMs to domains such as Wikipedia and Medical datasets.
A novel training protocol, SeND, and an unsupervised hallucination detection metric, EES, improve the reliability and factual accuracy of large language models.
large language models hallucinations Pythia suite SEnsitive Neuron Dropout SeND unsupervised hallucination detection Efficient EigenScore EES variance reduction sensitive neurons