SelfCite: Self-Supervised Alignment for Context Attribution in Large
  Language Models
We introduce SelfCite, a novel self-supervised approach that aligns LLMs to
generate high-quality, fine-grained, sentence-level citations for the
statements in their generated responses. Instead of only relying on costly and
labor-intensive annotations, SelfCite leverages a reward signal provided by the
LLM itself through context ablation: If a citation is necessary, removing the
cited text from the context should prevent the same response; if sufficient,
retaining the cited text alone should preserve the same response. This reward
can guide the inference-time best-of-N sampling strategy to improve citation
quality significantly, as well as be used in preference optimization to
directly fine-tune the models for generating better citations. The
effectiveness of SelfCite is demonstrated by increasing citation F1 up to 5.3
points on the LongBench-Cite benchmark across five long-form question answering
tasks.
67aeac4f2d48d9bf7728334e
Yung-Sung Chuang
extracted_confirmed
2025-02-14T02:37:32.909Z
5df84571da6d0311fd3d5407
https://cdn-avatars.huggingface.co/v1/production/uploads/1650651305661-5df84571da6d0311fd3d5407.png
Yung-Sung Chuang
user
voidism
67aeac4f2d48d9bf7728334f
Benjamin Cohen-Wang
claimed_verified
2025-02-14T08:01:17.696Z
639aaf82a4c528850bba2bfe
https://cdn-avatars.huggingface.co/v1/production/uploads/639aaf82a4c528850bba2bfe/nn23r8bsNiOJzVUxAPfo7.png
Benjamin Cohen-Wang
user
bencw
67aeac4f2d48d9bf77283350
Shannon Zejiang Shen
claimed_verified
2025-03-13T08:25:06.560Z
613f897ffbfd59f147a88c81
https://cdn-avatars.huggingface.co/v1/production/uploads/1659455519119-613f897ffbfd59f147a88c81.jpeg
Shannon Shen
user
shannons
67aeac4f2d48d9bf77283351
Zhaofeng Wu
claimed_verified
2025-02-14T08:01:19.691Z
6351712b40dffad651f128c7
/avatars/87708c86c1baef548ef556f5d32dca71.svg
Zhaofeng Wu
user
ZhaofengWu
67aeac4f2d48d9bf77283352
Hu Xu
67aeac4f2d48d9bf77283353
Xi Victoria Lin
67aeac4f2d48d9bf77283354
James Glass
67aeac4f2d48d9bf77283355
Shang-Wen Li
extracted_confirmed
2025-08-01T16:38:31.187Z
64e3a7b8e12618b261fff364
/avatars/860a5011f150537eb923f76d354f1bc5.svg
Shang-Wen Daniel Li
user
swdanielli
67aeac4f2d48d9bf77283356
Wen-tau Yih
SelfCite: Self-Supervised Alignment for Context Attribution in Large
  Language Models
We introduce SelfCite, a novel self-supervised approach that aligns LLMs to
generate high-quality, fine-grained, sentence-level citations for the
statements in their generated responses. Instead of only relying on costly and
labor-intensive annotations, SelfCite leverages a reward signal provided by the
LLM itself through context ablation: If a citation is necessary, removing the
cited text from the context should prevent the same response; if sufficient,
retaining the cited text alone should preserve the same response. This reward
can guide the inference-time best-of-N sampling strategy to improve citation
quality significantly, as well as be used in preference optimization to
directly fine-tune the models for generating better citations. The
effectiveness of SelfCite is demonstrated by increasing citation F1 up to 5.3
points on the LongBench-Cite benchmark across five long-form question answering
tasks.
SelfCite is a self-supervised method that uses context ablation to align LLMs for generating high-quality, sentence-level citations, improving citation quality and F1 scores on long-form question answering tasks.
self-supervised LLMs sentence-level citations context ablation reward signal best-of-N sampling preference optimization citation F1 LongBench-Cite