Trusted Source Alignment in Large Language Models
Large language models (LLMs) are trained on web-scale corpora that inevitably
include contradictory factual information from sources of varying reliability.
In this paper, we propose measuring an LLM property called trusted source
alignment (TSA): the model's propensity to align with content produced by
trusted publishers in the face of uncertainty or controversy. We present
FactCheckQA, a TSA evaluation dataset based on a corpus of fact checking
articles. We describe a simple protocol for evaluating TSA and offer a detailed
analysis of design considerations including response extraction, claim
contextualization, and bias in prompt formulation. Applying the protocol to
PaLM-2, we find that as we scale up the model size, the model performance on
FactCheckQA improves from near-random to up to 80% balanced accuracy in
aligning with trusted sources.
6552ec71b2d47c6675c991c3
Vasilisa Bashlovkina
6552ec71b2d47c6675c991c4
Zhaobin Kuang
6552ec71b2d47c6675c991c5
Riley Matthews
6552ec71b2d47c6675c991c6
Edward Clifford
6552ec71b2d47c6675c991c7
Yennie Jun
admin_assigned
2023-11-14T11:06:54.076Z
60b1100b5b0f651abe90a474
/avatars/64ac849a0022aa4b4debf7a2d1629a09.svg
yennie jun
user
yenniejun
6552ec71b2d47c6675c991c8
William W. Cohen
6552ec71b2d47c6675c991c9
Simon Baumgartner
claimed_verified
2023-11-27T08:43:24.824Z
6560f697e0a7720b6ae377bc
https://cdn-avatars.huggingface.co/v1/production/uploads/6560f697e0a7720b6ae377bc/8b-p-lQ7KV_VygBJ6Vf3_.jpeg
Simon Baumgartner
user
sens3
Trusted Source Alignment in Large Language Models
Large language models (LLMs) are trained on web-scale corpora that inevitably
include contradictory factual information from sources of varying reliability.
In this paper, we propose measuring an LLM property called trusted source
alignment (TSA): the model's propensity to align with content produced by
trusted publishers in the face of uncertainty or controversy. We present
FactCheckQA, a TSA evaluation dataset based on a corpus of fact checking
articles. We describe a simple protocol for evaluating TSA and offer a detailed
analysis of design considerations including response extraction, claim
contextualization, and bias in prompt formulation. Applying the protocol to
PaLM-2, we find that as we scale up the model size, the model performance on
FactCheckQA improves from near-random to up to 80% balanced accuracy in
aligning with trusted sources.
A new evaluation metric called trusted source alignment (TSA) is introduced and measured for large language models to assess their preference for content from trusted publishers over contradicting information.
large language models (LLMs) trusted source alignment (TSA) fact checking articles response extraction claim contextualization prompt formulation PaLM-2 balanced accuracy