title: Truth Neurons
summary: Despite their remarkable success and deployment across diverse workflows,
  language models sometimes produce untruthful responses. Our limited
  understanding of how truthfulness is mechanistically encoded within these
  models jeopardizes their reliability and safety. In this paper, we propose a
  method for identifying representations of truthfulness at the neuron level. We
  show that language models contain truth neurons, which encode truthfulness in a
  subject-agnostic manner. Experiments conducted across models of varying scales
  validate the existence of truth neurons, confirming that the encoding of
  truthfulness at the neuron level is a property shared by many language models.
  The distribution patterns of truth neurons over layers align with prior
  findings on the geometry of truthfulness. Selectively suppressing the
  activations of truth neurons found through the TruthfulQA dataset degrades
  performance both on TruthfulQA and on other benchmarks, showing that the
  truthfulness mechanisms are not tied to a specific dataset. Our results offer
  novel insights into the mechanisms underlying truthfulness in language models
  and highlight potential directions toward improving their trustworthiness and
  reliability.
paper_authors._id: 682d31dd17608739046e1169
paper_authors.name: Haohang Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T10:29:09.565Z
paper_authors._id: 634cabd104491d9f7111eea3
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/634cabd104491d9f7111eea3/JoqlugwfD1aGkd-wZTmP7.jpeg
paper_authors.fullname: Haohang Li
paper_authors.type: user
paper_authors.user: Acatsama
paper_authors._id: 682d31dd17608739046e116a
paper_authors.name: Yupeng Cao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T10:29:23.338Z
paper_authors._id: 62dd8f328456396d4f8aa894
paper_authors.avatarUrl: /avatars/af8f5dc7ff937e3e849ecdfd9ca4750b.svg
paper_authors.fullname: Yupeng Cao
paper_authors.type: user
paper_authors.user: YupengCao
paper_authors._id: 682d31dd17608739046e116b
paper_authors.name: Yangyang Yu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T10:29:31.615Z
paper_authors._id: 64f757c6016d60f3199ef5e6
paper_authors.avatarUrl: /avatars/2659ba698081265d0480b08161718013.svg
paper_authors.fullname: Yangyang Yu
paper_authors.type: user
paper_authors.user: ShirleyY
paper_authors._id: 682d31dd17608739046e116c
paper_authors.name: Jordan W. Suchow
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T10:29:41.392Z
paper_authors._id: 631a8c9e0867652f538a64e5
paper_authors.avatarUrl: /avatars/09e1cef9e7d6f7c51579448d4605850e.svg
paper_authors.fullname: Jordan Suchow
paper_authors.type: user
paper_authors.user: jordansuchow
paper_authors._id: 682d31dd17608739046e116d
paper_authors.name: Zining Zhu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-21T10:29:50.430Z
paper_authors._id: 62d63a9dc5ada8ef841b4787
paper_authors.avatarUrl: /avatars/a79a4ac07984d9a8623c99bdce9add54.svg
paper_authors.fullname: Zining Zhu
paper_authors.type: user
paper_authors.user: ZiningZhu
paper_title: Truth Neurons
paper_summary: Despite their remarkable success and deployment across diverse workflows,
  language models sometimes produce untruthful responses. Our limited
  understanding of how truthfulness is mechanistically encoded within these
  models jeopardizes their reliability and safety. In this paper, we propose a
  method for identifying representations of truthfulness at the neuron level. We
  show that language models contain truth neurons, which encode truthfulness in a
  subject-agnostic manner. Experiments conducted across models of varying scales
  validate the existence of truth neurons, confirming that the encoding of
  truthfulness at the neuron level is a property shared by many language models.
  The distribution patterns of truth neurons over layers align with prior
  findings on the geometry of truthfulness. Selectively suppressing the
  activations of truth neurons found through the TruthfulQA dataset degrades
  performance both on TruthfulQA and on other benchmarks, showing that the
  truthfulness mechanisms are not tied to a specific dataset. Our results offer
  novel insights into the mechanisms underlying truthfulness in language models
  and highlight potential directions toward improving their trustworthiness and
  reliability.
paper_ai_summary: Truth neurons encode truthfulness in language models at a subject-agnostic level, and suppressing these neurons degrades performance across benchmarks.
paper_ai_keywords: truth neurons language models truthfulness neuron level geometry of truthfulness TruthfulQA dataset