title: LLMs Know More Than They Show: On the Intrinsic Representation of LLM
    Hallucinations
summary: Large language models (LLMs) often produce errors, including factual
  inaccuracies, biases, and reasoning failures, collectively referred to as
  "hallucinations". Recent studies have demonstrated that LLMs' internal states
  encode information regarding the truthfulness of their outputs, and that this
  information can be utilized to detect errors. In this work, we show that the
  internal representations of LLMs encode much more information about
  truthfulness than previously recognized. We first discover that the
  truthfulness information is concentrated in specific tokens, and leveraging
  this property significantly enhances error detection performance. Yet, we show
  that such error detectors fail to generalize across datasets, implying that --
  contrary to prior claims -- truthfulness encoding is not universal but rather
  multifaceted. Next, we show that internal representations can also be used for
  predicting the types of errors the model is likely to make, facilitating the
  development of tailored mitigation strategies. Lastly, we reveal a discrepancy
  between LLMs' internal encoding and external behavior: they may encode the
  correct answer, yet consistently generate an incorrect one. Taken together,
  these insights deepen our understanding of LLM errors from the model's internal
  perspective, which can guide future research on enhancing error analysis and
  mitigation.
paper_authors._id: 6703267ec265fe2565c140da
paper_authors.name: Hadas Orgad
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-08T09:09:44.396Z
paper_authors._id: 60f82853c53e95176a7c6d45
paper_authors.avatarUrl: /avatars/16f6ea944a014af6ebe60499f3460784.svg
paper_authors.fullname: Hadas Orgad
paper_authors.type: user
paper_authors.user: hadasor
paper_authors._id: 6703267ec265fe2565c140db
paper_authors.name: Michael Toker
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-08T09:10:10.493Z
paper_authors._id: 61c865e3d3702a3bbf50bc04
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/61c865e3d3702a3bbf50bc04/eGgcyeUnOHz0hyap5vhvr.jpeg
paper_authors.fullname: Michael Toker
paper_authors.type: user
paper_authors.user: tokeron
paper_authors._id: 6703267ec265fe2565c140dc
paper_authors.name: Zorik Gekhman
paper_authors._id: 6703267ec265fe2565c140dd
paper_authors.name: Roi Reichart
paper_authors._id: 6703267ec265fe2565c140de
paper_authors.name: Idan Szpektor
paper_authors._id: 6703267ec265fe2565c140df
paper_authors.name: Hadas Kotek
paper_authors._id: 6703267ec265fe2565c140e0
paper_authors.name: Yonatan Belinkov
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-08T09:10:41.670Z
paper_authors._id: 614c57f1ee44bcfe57b366d6
paper_authors.avatarUrl: /avatars/186a9aed84681246f48ed2a012c50def.svg
paper_authors.fullname: Yonatan Belinkov
paper_authors.type: user
paper_authors.user: belinkov
paper_title: LLMs Know More Than They Show: On the Intrinsic Representation of LLM
    Hallucinations
paper_summary: Large language models (LLMs) often produce errors, including factual
  inaccuracies, biases, and reasoning failures, collectively referred to as
  "hallucinations". Recent studies have demonstrated that LLMs' internal states
  encode information regarding the truthfulness of their outputs, and that this
  information can be utilized to detect errors. In this work, we show that the
  internal representations of LLMs encode much more information about
  truthfulness than previously recognized. We first discover that the
  truthfulness information is concentrated in specific tokens, and leveraging
  this property significantly enhances error detection performance. Yet, we show
  that such error detectors fail to generalize across datasets, implying that --
  contrary to prior claims -- truthfulness encoding is not universal but rather
  multifaceted. Next, we show that internal representations can also be used for
  predicting the types of errors the model is likely to make, facilitating the
  development of tailored mitigation strategies. Lastly, we reveal a discrepancy
  between LLMs' internal encoding and external behavior: they may encode the
  correct answer, yet consistently generate an incorrect one. Taken together,
  these insights deepen our understanding of LLM errors from the model's internal
  perspective, which can guide future research on enhancing error analysis and
  mitigation.
paper_ai_summary: LLMs internally encode detailed truthfulness information about their outputs, which can be used to detect and predict errors but shows variability across datasets and does not always align with the final output.
paper_ai_keywords: LLMs hallucinations internal states truthfulness information error detection generalization error types error mitigation internal encoding external behavior