title: Trusted Source Alignment in Large Language Models
summary: Large language models (LLMs) are trained on web-scale corpora that inevitably
  include contradictory factual information from sources of varying reliability.
  In this paper, we propose measuring an LLM property called trusted source
  alignment (TSA): the model's propensity to align with content produced by
  trusted publishers in the face of uncertainty or controversy. We present
  FactCheckQA, a TSA evaluation dataset based on a corpus of fact checking
  articles. We describe a simple protocol for evaluating TSA and offer a detailed
  analysis of design considerations including response extraction, claim
  contextualization, and bias in prompt formulation. Applying the protocol to
  PaLM-2, we find that as we scale up the model size, the model performance on
  FactCheckQA improves from near-random to up to 80% balanced accuracy in
  aligning with trusted sources.
paper_authors._id: 6552ec71b2d47c6675c991c3
paper_authors.name: Vasilisa Bashlovkina
paper_authors._id: 6552ec71b2d47c6675c991c4
paper_authors.name: Zhaobin Kuang
paper_authors._id: 6552ec71b2d47c6675c991c5
paper_authors.name: Riley Matthews
paper_authors._id: 6552ec71b2d47c6675c991c6
paper_authors.name: Edward Clifford
paper_authors._id: 6552ec71b2d47c6675c991c7
paper_authors.name: Yennie Jun
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-14T11:06:54.076Z
paper_authors._id: 60b1100b5b0f651abe90a474
paper_authors.avatarUrl: /avatars/64ac849a0022aa4b4debf7a2d1629a09.svg
paper_authors.fullname: yennie jun
paper_authors.type: user
paper_authors.user: yenniejun
paper_authors._id: 6552ec71b2d47c6675c991c8
paper_authors.name: William W. Cohen
paper_authors._id: 6552ec71b2d47c6675c991c9
paper_authors.name: Simon Baumgartner
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-11-27T08:43:24.824Z
paper_authors._id: 6560f697e0a7720b6ae377bc
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6560f697e0a7720b6ae377bc/8b-p-lQ7KV_VygBJ6Vf3_.jpeg
paper_authors.fullname: Simon Baumgartner
paper_authors.type: user
paper_authors.user: sens3
paper_title: Trusted Source Alignment in Large Language Models
paper_summary: Large language models (LLMs) are trained on web-scale corpora that inevitably
  include contradictory factual information from sources of varying reliability.
  In this paper, we propose measuring an LLM property called trusted source
  alignment (TSA): the model's propensity to align with content produced by
  trusted publishers in the face of uncertainty or controversy. We present
  FactCheckQA, a TSA evaluation dataset based on a corpus of fact checking
  articles. We describe a simple protocol for evaluating TSA and offer a detailed
  analysis of design considerations including response extraction, claim
  contextualization, and bias in prompt formulation. Applying the protocol to
  PaLM-2, we find that as we scale up the model size, the model performance on
  FactCheckQA improves from near-random to up to 80% balanced accuracy in
  aligning with trusted sources.
paper_ai_keywords: trusted source alignment (TSA) fact checking articles response extraction claim contextualization bias in prompt formulation