title: Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of
    Language Models
summary: We investigate the internal behavior of Transformer-based Large Language
  Models (LLMs) when they generate factually incorrect text. We propose modeling
  factual queries as Constraint Satisfaction Problems and use this framework to
  investigate how the model interacts internally with factual constraints.
  Specifically, we discover a strong positive relation between the model's
  attention to constraint tokens and the factual accuracy of its responses. In
  our curated suite of 11 datasets with over 40,000 prompts, we study the task of
  predicting factual errors with the Llama-2 family across all scales (7B, 13B,
  70B). We propose SAT Probe, a method probing self-attention patterns, that can
  predict constraint satisfaction and factual errors, and allows early error
  identification. The approach and findings demonstrate how using the mechanistic
  understanding of factuality in LLMs can enhance reliability.
paper_authors._id: 65139c88bd21724fd44f1740
paper_authors.name: Mert Yuksekgonul
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-06-12T03:35:58.562Z
paper_authors._id: 61eb17ac8ae3ccab3b3da97b
paper_authors.avatarUrl: /avatars/9c91c33a0744eae1454e242d51295fbc.svg
paper_authors.fullname: Mert
paper_authors.type: user
paper_authors.user: merty
paper_authors._id: 65139c88bd21724fd44f1741
paper_authors.name: Varun Chandrasekaran
paper_authors._id: 65139c88bd21724fd44f1742
paper_authors.name: Erik Jones
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-27T10:19:35.272Z
paper_authors._id: 63518b69d95a8aa0c2247488
paper_authors.avatarUrl: /avatars/8ab0cb1e4e5d66c0018e22df035a6a9e.svg
paper_authors.fullname: Erik Jones
paper_authors.type: user
paper_authors.user: erikjones313
paper_authors._id: 65139c88bd21724fd44f1743
paper_authors.name: Suriya Gunasekar
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-27T10:19:42.961Z
paper_authors._id: 63e1b4f77fbb6ae4d4f36aa4
paper_authors.avatarUrl: /avatars/7febc332d393e0bb94255104527ffe2a.svg
paper_authors.fullname: Gunasekar
paper_authors.type: user
paper_authors.user: suriyagunasekar
paper_authors._id: 65139c88bd21724fd44f1744
paper_authors.name: Ranjita Naik
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-27T10:19:55.434Z
paper_authors._id: 63d8512207cd1aa3c4a1a835
paper_authors.avatarUrl: /avatars/cd5056c920a6b18eb93e2e9d0c5dc7c4.svg
paper_authors.fullname: Ranjita Naik
paper_authors.type: user
paper_authors.user: ranjita
paper_authors._id: 65139c88bd21724fd44f1745
paper_authors.name: Hamid Palangi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-09-27T10:20:03.661Z
paper_authors._id: 6189ddd0992df2640e3e7d40
paper_authors.avatarUrl: /avatars/925f2308fc412ae352b57a1b71815028.svg
paper_authors.fullname: Hamid Palangi
paper_authors.type: user
paper_authors.user: hamidpalangi
paper_authors._id: 65139c88bd21724fd44f1746
paper_authors.name: Ece Kamar
paper_authors._id: 65139c88bd21724fd44f1747
paper_authors.name: Besmira Nushi
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-02T07:00:24.772Z
paper_authors._id: 638ee2ed848625f9fad11034
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1670309301345-638ee2ed848625f9fad11034.jpeg
paper_authors.fullname: Besmira Nushi
paper_authors.type: user
paper_authors.user: nushib
paper_title: Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of
    Language Models
paper_summary: We investigate the internal behavior of Transformer-based Large Language
  Models (LLMs) when they generate factually incorrect text. We propose modeling
  factual queries as Constraint Satisfaction Problems and use this framework to
  investigate how the model interacts internally with factual constraints.
  Specifically, we discover a strong positive relation between the model's
  attention to constraint tokens and the factual accuracy of its responses. In
  our curated suite of 11 datasets with over 40,000 prompts, we study the task of
  predicting factual errors with the Llama-2 family across all scales (7B, 13B,
  70B). We propose SAT Probe, a method probing self-attention patterns, that can
  predict constraint satisfaction and factual errors, and allows early error
  identification. The approach and findings demonstrate how using the mechanistic
  understanding of factuality in LLMs can enhance reliability.
paper_ai_keywords: Transformer-based Large Language Models Constraint Satisfaction Problems attention constraint tokens factual accuracy self-attention patterns SAT Probe early error identification mechanistic understanding reliability