title: SelfCite: Self-Supervised Alignment for Context Attribution in Large
    Language Models
summary: We introduce SelfCite, a novel self-supervised approach that aligns LLMs to
  generate high-quality, fine-grained, sentence-level citations for the
  statements in their generated responses. Instead of only relying on costly and
  labor-intensive annotations, SelfCite leverages a reward signal provided by the
  LLM itself through context ablation: If a citation is necessary, removing the
  cited text from the context should prevent the same response; if sufficient,
  retaining the cited text alone should preserve the same response. This reward
  can guide the inference-time best-of-N sampling strategy to improve citation
  quality significantly, as well as be used in preference optimization to
  directly fine-tune the models for generating better citations. The
  effectiveness of SelfCite is demonstrated by increasing citation F1 up to 5.3
  points on the LongBench-Cite benchmark across five long-form question answering
  tasks.
paper_authors._id: 67aeac4f2d48d9bf7728334e
paper_authors.name: Yung-Sung Chuang
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-02-14T02:37:32.909Z
paper_authors._id: 5df84571da6d0311fd3d5407
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1650651305661-5df84571da6d0311fd3d5407.png
paper_authors.fullname: Yung-Sung Chuang
paper_authors.type: user
paper_authors.user: voidism
paper_authors._id: 67aeac4f2d48d9bf7728334f
paper_authors.name: Benjamin Cohen-Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-14T08:01:17.696Z
paper_authors._id: 639aaf82a4c528850bba2bfe
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/639aaf82a4c528850bba2bfe/nn23r8bsNiOJzVUxAPfo7.png
paper_authors.fullname: Benjamin Cohen-Wang
paper_authors.type: user
paper_authors.user: bencw
paper_authors._id: 67aeac4f2d48d9bf77283350
paper_authors.name: Shannon Zejiang Shen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-13T08:25:06.560Z
paper_authors._id: 613f897ffbfd59f147a88c81
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1659455519119-613f897ffbfd59f147a88c81.jpeg
paper_authors.fullname: Shannon Shen
paper_authors.type: user
paper_authors.user: shannons
paper_authors._id: 67aeac4f2d48d9bf77283351
paper_authors.name: Zhaofeng Wu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-14T08:01:19.691Z
paper_authors._id: 6351712b40dffad651f128c7
paper_authors.avatarUrl: /avatars/87708c86c1baef548ef556f5d32dca71.svg
paper_authors.fullname: Zhaofeng Wu
paper_authors.type: user
paper_authors.user: ZhaofengWu
paper_authors._id: 67aeac4f2d48d9bf77283352
paper_authors.name: Hu Xu
paper_authors._id: 67aeac4f2d48d9bf77283353
paper_authors.name: Xi Victoria Lin
paper_authors._id: 67aeac4f2d48d9bf77283354
paper_authors.name: James Glass
paper_authors._id: 67aeac4f2d48d9bf77283355
paper_authors.name: Shang-Wen Li
paper_authors._id: 67aeac4f2d48d9bf77283356
paper_authors.name: Wen-tau Yih
paper_title: SelfCite: Self-Supervised Alignment for Context Attribution in Large
    Language Models
paper_summary: We introduce SelfCite, a novel self-supervised approach that aligns LLMs to
  generate high-quality, fine-grained, sentence-level citations for the
  statements in their generated responses. Instead of only relying on costly and
  labor-intensive annotations, SelfCite leverages a reward signal provided by the
  LLM itself through context ablation: If a citation is necessary, removing the
  cited text from the context should prevent the same response; if sufficient,
  retaining the cited text alone should preserve the same response. This reward
  can guide the inference-time best-of-N sampling strategy to improve citation
  quality significantly, as well as be used in preference optimization to
  directly fine-tune the models for generating better citations. The
  effectiveness of SelfCite is demonstrated by increasing citation F1 up to 5.3
  points on the LongBench-Cite benchmark across five long-form question answering
  tasks.
paper_ai_keywords: Self-supervised approach SelfCite Large Language Models (LLMs) Sentence-level citations Rewards signal Context ablations Best-of-N sampling strategy LongBench-Cite benchmark Long-form question answering tasks Citation F1 score