title: Multiple Choice Questions: Reasoning Makes Large Language Models (LLMs)
    More Self-Confident Even When They Are Wrong
summary: One of the most widely used methods to evaluate LLMs are Multiple Choice
  Question (MCQ) tests. MCQ benchmarks enable the testing of LLM knowledge on
  almost any topic at scale as the results can be processed automatically. To
  help the LLM answer, a few examples called few shots can be included in the
  prompt. Moreover, the LLM can be asked to answer the question directly with the
  selected option or to first provide the reasoning and then the selected answer,
  which is known as chain of thought. In addition to checking whether the
  selected answer is correct, the evaluation can look at the LLM-estimated
  probability of its response as an indication of the confidence of the LLM in
  the response. In this paper, we study how the LLM confidence in its answer
  depends on whether the model has been asked to answer directly or to provide
  the reasoning before answering. The results of the evaluation of questions on a
  wide range of topics in seven different models show that LLMs are more
  confident in their answers when they provide reasoning before the answer. This
  occurs regardless of whether the selected answer is correct. Our hypothesis is
  that this behavior is due to the reasoning that modifies the probability of the
  selected answer, as the LLM predicts the answer based on the input question and
  the reasoning that supports the selection made. Therefore, LLM estimated
  probabilities seem to have intrinsic limitations that should be understood in
  order to use them in evaluation procedures. Interestingly, the same behavior
  has been observed in humans, for whom explaining an answer increases confidence
  in its correctness.
paper_authors._id: 678e1b151a99a49b98056e1c
paper_authors.name: Tairan Fu
paper_authors._id: 678e1b151a99a49b98056e1d
paper_authors.name: Javier Conde
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-21T10:51:53.077Z
paper_authors._id: 66852115f8c3f32152346733
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/66852115f8c3f32152346733/EuApIooUvMesCSr27Auze.jpeg
paper_authors.fullname: Javier Conde
paper_authors.type: user
paper_authors.user: javicond3
paper_authors._id: 678e1b151a99a49b98056e1e
paper_authors.name: Gonzalo Martínez
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-20T16:59:39.580Z
paper_authors._id: 64f31365ed48e3bb9c487d5d
paper_authors.avatarUrl: /avatars/979c1979eadbd4529c95b925bbb58d78.svg
paper_authors.fullname: Gonzalo
paper_authors.type: user
paper_authors.user: gonzmart
paper_authors._id: 678e1b151a99a49b98056e1f
paper_authors.name: María Grandury
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-01-20T10:35:14.566Z
paper_authors._id: 5f9c00a5777efc07d7f1e4be
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1665073337782-5f9c00a5777efc07d7f1e4be.png
paper_authors.fullname: María Grandury
paper_authors.type: user
paper_authors.user: mariagrandury
paper_authors._id: 678e1b151a99a49b98056e20
paper_authors.name: Pedro Reviriego
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-20T13:16:45.528Z
paper_authors._id: 6574f66b06fdcd4ca9491299
paper_authors.avatarUrl: /avatars/e31821949d75efb750ab2d9ebe12b9a8.svg
paper_authors.fullname: pedro reviriego
paper_authors.type: user
paper_authors.user: reviriego
paper_title: Multiple Choice Questions: Reasoning Makes Large Language Models (LLMs)
    More Self-Confident Even When They Are Wrong
paper_summary: One of the most widely used methods to evaluate LLMs are Multiple Choice
  Question (MCQ) tests. MCQ benchmarks enable the testing of LLM knowledge on
  almost any topic at scale as the results can be processed automatically. To
  help the LLM answer, a few examples called few shots can be included in the
  prompt. Moreover, the LLM can be asked to answer the question directly with the
  selected option or to first provide the reasoning and then the selected answer,
  which is known as chain of thought. In addition to checking whether the
  selected answer is correct, the evaluation can look at the LLM-estimated
  probability of its response as an indication of the confidence of the LLM in
  the response. In this paper, we study how the LLM confidence in its answer
  depends on whether the model has been asked to answer directly or to provide
  the reasoning before answering. The results of the evaluation of questions on a
  wide range of topics in seven different models show that LLMs are more
  confident in their answers when they provide reasoning before the answer. This
  occurs regardless of whether the selected answer is correct. Our hypothesis is
  that this behavior is due to the reasoning that modifies the probability of the
  selected answer, as the LLM predicts the answer based on the input question and
  the reasoning that supports the selection made. Therefore, LLM estimated
  probabilities seem to have intrinsic limitations that should be understood in
  order to use them in evaluation procedures. Interestingly, the same behavior
  has been observed in humans, for whom explaining an answer increases confidence
  in its correctness.
paper_ai_keywords: dense language models (LLMs) Multiple Choice Question (MCQ) tests MCQ benchmarks few-shot examples chain of thought LLM confidence probability of response intrinsic limitations