Llamas Know What GPTs Don't Show: Surrogate Models for Confidence
  Estimation
To maintain user trust, large language models (LLMs) should signal low
confidence on examples where they are incorrect, instead of misleading the
user. The standard approach of estimating confidence is to use the softmax
probabilities of these models, but as of November 2023, state-of-the-art LLMs
such as GPT-4 and Claude-v1.3 do not provide access to these probabilities. We
first study eliciting confidence linguistically -- asking an LLM for its
confidence in its answer -- which performs reasonably (80.5% AUC on GPT-4
averaged across 12 question-answering datasets -- 7% above a random baseline)
but leaves room for improvement. We then explore using a surrogate confidence
model -- using a model where we do have probabilities to evaluate the original
model's confidence in a given question. Surprisingly, even though these
probabilities come from a different and often weaker model, this method leads
to higher AUC than linguistic confidences on 9 out of 12 datasets. Our best
method composing linguistic confidences and surrogate model probabilities gives
state-of-the-art confidence estimates on all 12 datasets (84.6% average AUC on
GPT-4).
6555aa48d5a72476aef11f0b
Vaishnavi Shrivastava
admin_assigned
2023-11-16T10:18:08.677Z
63b75a016fc56e43c3c15980
https://cdn-avatars.huggingface.co/v1/production/uploads/1672960383634-noauth.jpeg
Vaishnavi Shrivastava
user
vshrivas
6555aa48d5a72476aef11f0c
Percy Liang
admin_assigned
2023-11-16T10:18:15.825Z
6409651b9e9f790c905b2335
/avatars/1fb8c80b60f21f65a0a027319101f236.svg
Percy Liang
user
percyliang
6555aa48d5a72476aef11f0d
Ananya Kumar
Llamas Know What GPTs Don't Show: Surrogate Models for Confidence
  Estimation
To maintain user trust, large language models (LLMs) should signal low
confidence on examples where they are incorrect, instead of misleading the
user. The standard approach of estimating confidence is to use the softmax
probabilities of these models, but as of November 2023, state-of-the-art LLMs
such as GPT-4 and Claude-v1.3 do not provide access to these probabilities. We
first study eliciting confidence linguistically -- asking an LLM for its
confidence in its answer -- which performs reasonably (80.5% AUC on GPT-4
averaged across 12 question-answering datasets -- 7% above a random baseline)
but leaves room for improvement. We then explore using a surrogate confidence
model -- using a model where we do have probabilities to evaluate the original
model's confidence in a given question. Surprisingly, even though these
probabilities come from a different and often weaker model, this method leads
to higher AUC than linguistic confidences on 9 out of 12 datasets. Our best
method composing linguistic confidences and surrogate model probabilities gives
state-of-the-art confidence estimates on all 12 datasets (84.6% average AUC on
GPT-4).
A technique combining linguistic confidence and surrogate model probabilities yields the highest confidence estimates in state-of-the-art large language models.
large language models LLMs softmax probabilities AUC question-answering datasets surrogate confidence model confidence estimates