title: To Believe or Not to Believe Your LLM
summary: We explore uncertainty quantification in large language models (LLMs), with
  the goal to identify when uncertainty in responses given a query is large. We
  simultaneously consider both epistemic and aleatoric uncertainties, where the
  former comes from the lack of knowledge about the ground truth (such as about
  facts or the language), and the latter comes from irreducible randomness (such
  as multiple possible answers). In particular, we derive an
  information-theoretic metric that allows to reliably detect when only epistemic
  uncertainty is large, in which case the output of the model is unreliable. This
  condition can be computed based solely on the output of the model obtained
  simply by some special iterative prompting based on the previous responses.
  Such quantification, for instance, allows to detect hallucinations (cases when
  epistemic uncertainty is high) in both single- and multi-answer responses. This
  is in contrast to many standard uncertainty quantification strategies (such as
  thresholding the log-likelihood of a response) where hallucinations in the
  multi-answer case cannot be detected. We conduct a series of experiments which
  demonstrate the advantage of our formulation. Further, our investigations shed
  some light on how the probabilities assigned to a given output by an LLM can be
  amplified by iterative prompting, which might be of independent interest.
paper_authors._id: 665fe728ae03446a3da121cb
paper_authors.name: Yasin Abbasi Yadkori
paper_authors._id: 665fe728ae03446a3da121cc
paper_authors.name: Ilja Kuzborskij
paper_authors._id: 665fe728ae03446a3da121cd
paper_authors.name: András György
paper_authors._id: 665fe728ae03446a3da121ce
paper_authors.name: Csaba Szepesvári
paper_title: To Believe or Not to Believe Your LLM
paper_summary: We explore uncertainty quantification in large language models (LLMs), with
  the goal to identify when uncertainty in responses given a query is large. We
  simultaneously consider both epistemic and aleatoric uncertainties, where the
  former comes from the lack of knowledge about the ground truth (such as about
  facts or the language), and the latter comes from irreducible randomness (such
  as multiple possible answers). In particular, we derive an
  information-theoretic metric that allows to reliably detect when only epistemic
  uncertainty is large, in which case the output of the model is unreliable. This
  condition can be computed based solely on the output of the model obtained
  simply by some special iterative prompting based on the previous responses.
  Such quantification, for instance, allows to detect hallucinations (cases when
  epistemic uncertainty is high) in both single- and multi-answer responses. This
  is in contrast to many standard uncertainty quantification strategies (such as
  thresholding the log-likelihood of a response) where hallucinations in the
  multi-answer case cannot be detected. We conduct a series of experiments which
  demonstrate the advantage of our formulation. Further, our investigations shed
  some light on how the probabilities assigned to a given output by an LLM can be
  amplified by iterative prompting, which might be of independent interest.
paper_ai_keywords: uncertainty quantification large language models (LLMs) epistemic uncertainty aleatoric uncertainty information-theoretic metric iterative prompting hallucinations log-likelihood