ContextCite: Attributing Model Generation to Context
How do language models use information provided as context when generating a
response? Can we infer whether a particular generated statement is actually
grounded in the context, a misinterpretation, or fabricated? To help answer
these questions, we introduce the problem of context attribution: pinpointing
the parts of the context (if any) that led a model to generate a particular
statement. We then present ContextCite, a simple and scalable method for
context attribution that can be applied on top of any existing language model.
Finally, we showcase the utility of ContextCite through three applications: (1)
helping verify generated statements (2) improving response quality by pruning
the context and (3) detecting poisoning attacks. We provide code for
ContextCite at https://github.com/MadryLab/context-cite.
66d802899fcb6b494f44536f
Benjamin Cohen-Wang
admin_assigned
2024-09-04T09:52:18.420Z
639aaf82a4c528850bba2bfe
https://cdn-avatars.huggingface.co/v1/production/uploads/639aaf82a4c528850bba2bfe/nn23r8bsNiOJzVUxAPfo7.png
Benjamin Cohen-Wang
user
bencw
66d802899fcb6b494f445370
Harshay Shah
claimed_verified
2024-09-04T08:04:28.641Z
64b1a4f64dd3e24895daa236
https://cdn-avatars.huggingface.co/v1/production/uploads/64b1a4f64dd3e24895daa236/lzlZ4DBOw-YVspNttAEY3.jpeg
Harshay Shah
user
harshay
66d802899fcb6b494f445371
Kristian Georgiev
admin_assigned
2024-09-04T09:52:23.838Z
63b98550efe99543b34cd33d
https://cdn-avatars.huggingface.co/v1/production/uploads/63b98550efe99543b34cd33d/lsfiSVfQeKOypMh66c8pT.jpeg
Kristian Georgiev
user
krisgrg
66d802899fcb6b494f445372
Aleksander Madry
ContextCite: Attributing Model Generation to Context
How do language models use information provided as context when generating a
response? Can we infer whether a particular generated statement is actually
grounded in the context, a misinterpretation, or fabricated? To help answer
these questions, we introduce the problem of context attribution: pinpointing
the parts of the context (if any) that led a model to generate a particular
statement. We then present ContextCite, a simple and scalable method for
context attribution that can be applied on top of any existing language model.
Finally, we showcase the utility of ContextCite through three applications: (1)
helping verify generated statements (2) improving response quality by pruning
the context and (3) detecting poisoning attacks. We provide code for
ContextCite at https://github.com/MadryLab/context-cite.
ContextCite is a method for attributing parts of the context used by language models to generate statements, aiding in verification, improving response quality, and detecting poisoning attacks.
context attribution language models ContextCite generated statements response quality poisoning attacks