title: Document Attribution: Examining Citation Relationships using Large
    Language Models
summary: As Large Language Models (LLMs) are increasingly applied to document-based
  tasks - such as document summarization, question answering, and information
  extraction - where user requirements focus on retrieving information from
  provided documents rather than relying on the model's parametric knowledge,
  ensuring the trustworthiness and interpretability of these systems has become a
  critical concern. A central approach to addressing this challenge is
  attribution, which involves tracing the generated outputs back to their source
  documents. However, since LLMs can produce inaccurate or imprecise responses,
  it is crucial to assess the reliability of these citations.
    To tackle this, our work proposes two techniques. (1) A zero-shot approach
  that frames attribution as a straightforward textual entailment task. Our
  method using flan-ul2 demonstrates an improvement of 0.27% and 2.4% over the
  best baseline of ID and OOD sets of AttributionBench, respectively. (2) We also
  explore the role of the attention mechanism in enhancing the attribution
  process. Using a smaller LLM, flan-t5-small, the F1 scores outperform the
  baseline across almost all layers except layer 4 and layers 8 through 11.
paper_authors._id: 6822bfeaa371047fe1545a13
paper_authors.name: Vipula Rawte
paper_authors._id: 6822bfeaa371047fe1545a14
paper_authors.name: Ryan A. Rossi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-13T08:40:15.552Z
paper_authors._id: 62a3ab83e4dd6252344d27cd
paper_authors.avatarUrl: /avatars/7ca8510f70a58dc207b104240e30c35c.svg
paper_authors.fullname: Ryan A. Rossi
paper_authors.type: user
paper_authors.user: ryanrossi
paper_authors._id: 6822bfeaa371047fe1545a15
paper_authors.name: Franck Dernoncourt
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-13T08:11:40.296Z
paper_authors._id: 62c5947524171688a9feb992
paper_authors.avatarUrl: /avatars/5a151713b9eae8dc566f5957acee3475.svg
paper_authors.fullname: Franck Dernoncourt
paper_authors.type: user
paper_authors.user: Franck-Dernoncourt
paper_authors._id: 6822bfeaa371047fe1545a16
paper_authors.name: Nedim Lipka
paper_title: Document Attribution: Examining Citation Relationships using Large
    Language Models
paper_summary: As Large Language Models (LLMs) are increasingly applied to document-based
  tasks - such as document summarization, question answering, and information
  extraction - where user requirements focus on retrieving information from
  provided documents rather than relying on the model's parametric knowledge,
  ensuring the trustworthiness and interpretability of these systems has become a
  critical concern. A central approach to addressing this challenge is
  attribution, which involves tracing the generated outputs back to their source
  documents. However, since LLMs can produce inaccurate or imprecise responses,
  it is crucial to assess the reliability of these citations.
    To tackle this, our work proposes two techniques. (1) A zero-shot approach
  that frames attribution as a straightforward textual entailment task. Our
  method using flan-ul2 demonstrates an improvement of 0.27% and 2.4% over the
  best baseline of ID and OOD sets of AttributionBench, respectively. (2) We also
  explore the role of the attention mechanism in enhancing the attribution
  process. Using a smaller LLM, flan-t5-small, the F1 scores outperform the
  baseline across almost all layers except layer 4 and layers 8 through 11.
paper_ai_summary: The study proposes techniques to enhance attribution in Large Language Models (LLMs) for document-based tasks, using textual entailment and attention mechanisms to improve the reliability and trustworthiness of citations.
paper_ai_keywords: Large Language Models (LLMs) document summarization question answering information extraction attribution textual entailment flan-ul2 AttributionBench F1 scores attention mechanism