Lookback Lens: Detecting and Mitigating Contextual Hallucinations in
  Large Language Models Using Only Attention Maps
When asked to summarize articles or answer questions given a passage, large
language models (LLMs) can hallucinate details and respond with unsubstantiated
answers that are inaccurate with respect to the input context. This paper
describes a simple approach for detecting such contextual hallucinations. We
hypothesize that contextual hallucinations are related to the extent to which
an LLM attends to information in the provided context versus its own
generations. Based on this intuition, we propose a simple hallucination
detection model whose input features are given by the ratio of attention
weights on the context versus newly generated tokens (for each attention head).
We find that a linear classifier based on these lookback ratio features is as
effective as a richer detector that utilizes the entire hidden states of an LLM
or a text-based entailment model. The lookback ratio-based detector -- Lookback
Lens -- is found to transfer across tasks and even models, allowing a detector
that is trained on a 7B model to be applied (without retraining) to a larger
13B model. We further apply this detector to mitigate contextual
hallucinations, and find that a simple classifier-guided decoding approach is
able to reduce the amount of hallucination, for example by 9.6% in the XSum
summarization task.
668dead717676d7791bbaa43
Yung-Sung Chuang
extracted_confirmed
2024-07-10T01:59:40.696Z
5df84571da6d0311fd3d5407
https://cdn-avatars.huggingface.co/v1/production/uploads/1650651305661-5df84571da6d0311fd3d5407.png
Yung-Sung Chuang
user
voidism
668dead717676d7791bbaa44
Linlu Qiu
admin_assigned
2024-07-10T10:39:10.836Z
632ba760ca316c73cd400365
/avatars/e166f54ea69cdc4119f29318ace7f4db.svg
Linlu Qiu
user
linluqiu
668dead717676d7791bbaa45
Cheng-Yu Hsieh
668dead717676d7791bbaa46
Ranjay Krishna
admin_assigned
2024-07-10T10:40:22.603Z
66429868ab89e3a3a85668b0
/avatars/170e0daa454838deee2bf946f7118651.svg
Ranjay Krishna
user
ranjaykrishna
668dead717676d7791bbaa47
Yoon Kim
668dead717676d7791bbaa48
James Glass
admin_assigned
2024-07-10T10:40:47.357Z
62a793d33e101ec156cc58c4
/avatars/d3607cdb2cac1b2a3011d7441e6ea321.svg
James Glass
user
OmegaLittleBob
Lookback Lens: Detecting and Mitigating Contextual Hallucinations in
  Large Language Models Using Only Attention Maps
When asked to summarize articles or answer questions given a passage, large
language models (LLMs) can hallucinate details and respond with unsubstantiated
answers that are inaccurate with respect to the input context. This paper
describes a simple approach for detecting such contextual hallucinations. We
hypothesize that contextual hallucinations are related to the extent to which
an LLM attends to information in the provided context versus its own
generations. Based on this intuition, we propose a simple hallucination
detection model whose input features are given by the ratio of attention
weights on the context versus newly generated tokens (for each attention head).
We find that a linear classifier based on these lookback ratio features is as
effective as a richer detector that utilizes the entire hidden states of an LLM
or a text-based entailment model. The lookback ratio-based detector -- Lookback
Lens -- is found to transfer across tasks and even models, allowing a detector
that is trained on a 7B model to be applied (without retraining) to a larger
13B model. We further apply this detector to mitigate contextual
hallucinations, and find that a simple classifier-guided decoding approach is
able to reduce the amount of hallucination, for example by 9.6% in the XSum
summarization task.
A linear attention-based classifier reduces contextual hallucinations in language models by focusing on the ratio of attention to given context versus generated tokens.
attention weights contextual hallucinations linear classifier lookback ratio lookback lens classifier-guided decoding XSum summarization task