title: Language Models Prefer What They Know: Relative Confidence Estimation
    via Confidence Preferences
summary: Language models (LMs) should provide reliable confidence estimates to help
  users detect mistakes in their outputs and defer to human experts when
  necessary. Asking a language model to assess its confidence ("Score your
  confidence from 0-1.") is a natural way of evaluating its uncertainty. However,
  models struggle to provide absolute assessments of confidence (i.e. judging
  confidence in answering a question independent of other questions) and the
  coarse-grained scores they produce are not useful for evaluating the
  correctness of their answers. We propose relative confidence estimation, where
  we match up questions against each other and ask the model to make relative
  judgments of confidence ("Which question are you more confident in answering
  correctly?"). Treating each question as a "player" in a series of matchups
  against other questions and the model's preferences as match outcomes, we can
  use rank aggregation methods like Elo rating and Bradley-Terry to translate the
  model's confidence preferences into confidence scores. We evaluate relative
  confidence estimation against absolute confidence estimation and
  self-consistency confidence methods on five state-of-the-art LMs -- GPT-4,
  GPT-4o, Gemini 1.5 Pro, Claude 3.5 Sonnet, and Llama 3.1 405B -- across 14
  challenging STEM, social science, and commonsense reasoning question answering
  tasks. Our results demonstrate that relative confidence estimation consistently
  provides more reliable confidence scores than absolute confidence estimation,
  with average gains of 3.5% in selective classification AUC over direct absolute
  confidence estimation methods and 1.7% over self-consistency approaches across
  all models and datasets.
paper_authors._id: 67a27eb5d6a1524a1a6f048c
paper_authors.name: Vaishnavi Shrivastava
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-04T21:14:18.633Z
paper_authors._id: 63b75a016fc56e43c3c15980
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1672960383634-noauth.jpeg
paper_authors.fullname: Vaishnavi Shrivastava
paper_authors.type: user
paper_authors.user: vshrivas
paper_authors._id: 67a27eb5d6a1524a1a6f048d
paper_authors.name: Ananya Kumar
paper_authors._id: 67a27eb5d6a1524a1a6f048e
paper_authors.name: Percy Liang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-02-04T21:14:47.149Z
paper_authors._id: 6409651b9e9f790c905b2335
paper_authors.avatarUrl: /avatars/1fb8c80b60f21f65a0a027319101f236.svg
paper_authors.fullname: Percy Liang
paper_authors.type: user
paper_authors.user: percyliang
paper_title: Language Models Prefer What They Know: Relative Confidence Estimation
    via Confidence Preferences
paper_summary: Language models (LMs) should provide reliable confidence estimates to help
  users detect mistakes in their outputs and defer to human experts when
  necessary. Asking a language model to assess its confidence ("Score your
  confidence from 0-1.") is a natural way of evaluating its uncertainty. However,
  models struggle to provide absolute assessments of confidence (i.e. judging
  confidence in answering a question independent of other questions) and the
  coarse-grained scores they produce are not useful for evaluating the
  correctness of their answers. We propose relative confidence estimation, where
  we match up questions against each other and ask the model to make relative
  judgments of confidence ("Which question are you more confident in answering
  correctly?"). Treating each question as a "player" in a series of matchups
  against other questions and the model's preferences as match outcomes, we can
  use rank aggregation methods like Elo rating and Bradley-Terry to translate the
  model's confidence preferences into confidence scores. We evaluate relative
  confidence estimation against absolute confidence estimation and
  self-consistency confidence methods on five state-of-the-art LMs -- GPT-4,
  GPT-4o, Gemini 1.5 Pro, Claude 3.5 Sonnet, and Llama 3.1 405B -- across 14
  challenging STEM, social science, and commonsense reasoning question answering
  tasks. Our results demonstrate that relative confidence estimation consistently
  provides more reliable confidence scores than absolute confidence estimation,
  with average gains of 3.5% in selective classification AUC over direct absolute
  confidence estimation methods and 1.7% over self-consistency approaches across
  all models and datasets.