title: Lookback Lens: Detecting and Mitigating Contextual Hallucinations in
    Large Language Models Using Only Attention Maps
summary: When asked to summarize articles or answer questions given a passage, large
  language models (LLMs) can hallucinate details and respond with unsubstantiated
  answers that are inaccurate with respect to the input context. This paper
  describes a simple approach for detecting such contextual hallucinations. We
  hypothesize that contextual hallucinations are related to the extent to which
  an LLM attends to information in the provided context versus its own
  generations. Based on this intuition, we propose a simple hallucination
  detection model whose input features are given by the ratio of attention
  weights on the context versus newly generated tokens (for each attention head).
  We find that a linear classifier based on these lookback ratio features is as
  effective as a richer detector that utilizes the entire hidden states of an LLM
  or a text-based entailment model. The lookback ratio-based detector -- Lookback
  Lens -- is found to transfer across tasks and even models, allowing a detector
  that is trained on a 7B model to be applied (without retraining) to a larger
  13B model. We further apply this detector to mitigate contextual
  hallucinations, and find that a simple classifier-guided decoding approach is
  able to reduce the amount of hallucination, for example by 9.6% in the XSum
  summarization task.
paper_authors._id: 668dead717676d7791bbaa43
paper_authors.name: Yung-Sung Chuang
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-07-10T01:59:40.696Z
paper_authors._id: 5df84571da6d0311fd3d5407
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1650651305661-5df84571da6d0311fd3d5407.png
paper_authors.fullname: Yung-Sung Chuang
paper_authors.type: user
paper_authors.user: voidism
paper_authors._id: 668dead717676d7791bbaa44
paper_authors.name: Linlu Qiu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-10T10:39:10.836Z
paper_authors._id: 632ba760ca316c73cd400365
paper_authors.avatarUrl: /avatars/e166f54ea69cdc4119f29318ace7f4db.svg
paper_authors.fullname: Linlu Qiu
paper_authors.type: user
paper_authors.user: linluqiu
paper_authors._id: 668dead717676d7791bbaa45
paper_authors.name: Cheng-Yu Hsieh
paper_authors._id: 668dead717676d7791bbaa46
paper_authors.name: Ranjay Krishna
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-10T10:40:22.603Z
paper_authors._id: 66429868ab89e3a3a85668b0
paper_authors.avatarUrl: /avatars/170e0daa454838deee2bf946f7118651.svg
paper_authors.fullname: Ranjay Krishna
paper_authors.type: user
paper_authors.user: ranjaykrishna
paper_authors._id: 668dead717676d7791bbaa47
paper_authors.name: Yoon Kim
paper_authors._id: 668dead717676d7791bbaa48
paper_authors.name: James Glass
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-10T10:40:47.357Z
paper_authors._id: 62a793d33e101ec156cc58c4
paper_authors.avatarUrl: /avatars/d3607cdb2cac1b2a3011d7441e6ea321.svg
paper_authors.fullname: James Glass
paper_authors.type: user
paper_authors.user: OmegaLittleBob
paper_title: Lookback Lens: Detecting and Mitigating Contextual Hallucinations in
    Large Language Models Using Only Attention Maps
paper_summary: When asked to summarize articles or answer questions given a passage, large
  language models (LLMs) can hallucinate details and respond with unsubstantiated
  answers that are inaccurate with respect to the input context. This paper
  describes a simple approach for detecting such contextual hallucinations. We
  hypothesize that contextual hallucinations are related to the extent to which
  an LLM attends to information in the provided context versus its own
  generations. Based on this intuition, we propose a simple hallucination
  detection model whose input features are given by the ratio of attention
  weights on the context versus newly generated tokens (for each attention head).
  We find that a linear classifier based on these lookback ratio features is as
  effective as a richer detector that utilizes the entire hidden states of an LLM
  or a text-based entailment model. The lookback ratio-based detector -- Lookback
  Lens -- is found to transfer across tasks and even models, allowing a detector
  that is trained on a 7B model to be applied (without retraining) to a larger
  13B model. We further apply this detector to mitigate contextual
  hallucinations, and find that a simple classifier-guided decoding approach is
  able to reduce the amount of hallucination, for example by 9.6% in the XSum
  summarization task.
paper_ai_keywords: contextual hallucinations attention weights linear classifier lookback ratio hidden states text-based entailment model parameter-efficient fine-tuning classifier-guided decoding