Multiple Choice Questions: Reasoning Makes Large Language Models (LLMs)
  More Self-Confident Even When They Are Wrong
One of the most widely used methods to evaluate LLMs are Multiple Choice
Question (MCQ) tests. MCQ benchmarks enable the testing of LLM knowledge on
almost any topic at scale as the results can be processed automatically. To
help the LLM answer, a few examples called few shots can be included in the
prompt. Moreover, the LLM can be asked to answer the question directly with the
selected option or to first provide the reasoning and then the selected answer,
which is known as chain of thought. In addition to checking whether the
selected answer is correct, the evaluation can look at the LLM-estimated
probability of its response as an indication of the confidence of the LLM in
the response. In this paper, we study how the LLM confidence in its answer
depends on whether the model has been asked to answer directly or to provide
the reasoning before answering. The results of the evaluation of questions on a
wide range of topics in seven different models show that LLMs are more
confident in their answers when they provide reasoning before the answer. This
occurs regardless of whether the selected answer is correct. Our hypothesis is
that this behavior is due to the reasoning that modifies the probability of the
selected answer, as the LLM predicts the answer based on the input question and
the reasoning that supports the selection made. Therefore, LLM estimated
probabilities seem to have intrinsic limitations that should be understood in
order to use them in evaluation procedures. Interestingly, the same behavior
has been observed in humans, for whom explaining an answer increases confidence
in its correctness.
678e1b151a99a49b98056e1c
Tairan Fu
678e1b151a99a49b98056e1d
Javier Conde
claimed_verified
2025-01-21T10:51:53.077Z
66852115f8c3f32152346733
https://cdn-avatars.huggingface.co/v1/production/uploads/66852115f8c3f32152346733/EuApIooUvMesCSr27Auze.jpeg
Javier Conde
user
javicond3
678e1b151a99a49b98056e1e
Gonzalo Martínez
admin_assigned
2025-01-20T16:59:39.580Z
64f31365ed48e3bb9c487d5d
/avatars/979c1979eadbd4529c95b925bbb58d78.svg
Gonzalo
user
gonzmart
678e1b151a99a49b98056e1f
María Grandury
extracted_confirmed
2025-01-20T10:35:14.566Z
5f9c00a5777efc07d7f1e4be
https://cdn-avatars.huggingface.co/v1/production/uploads/1665073337782-5f9c00a5777efc07d7f1e4be.png
María Grandury
user
mariagrandury
678e1b151a99a49b98056e20
Pedro Reviriego
claimed_verified
2025-01-20T13:16:45.528Z
6574f66b06fdcd4ca9491299
/avatars/e31821949d75efb750ab2d9ebe12b9a8.svg
pedro reviriego
user
reviriego
Multiple Choice Questions: Reasoning Makes Large Language Models (LLMs)
  More Self-Confident Even When They Are Wrong
One of the most widely used methods to evaluate LLMs are Multiple Choice
Question (MCQ) tests. MCQ benchmarks enable the testing of LLM knowledge on
almost any topic at scale as the results can be processed automatically. To
help the LLM answer, a few examples called few shots can be included in the
prompt. Moreover, the LLM can be asked to answer the question directly with the
selected option or to first provide the reasoning and then the selected answer,
which is known as chain of thought. In addition to checking whether the
selected answer is correct, the evaluation can look at the LLM-estimated
probability of its response as an indication of the confidence of the LLM in
the response. In this paper, we study how the LLM confidence in its answer
depends on whether the model has been asked to answer directly or to provide
the reasoning before answering. The results of the evaluation of questions on a
wide range of topics in seven different models show that LLMs are more
confident in their answers when they provide reasoning before the answer. This
occurs regardless of whether the selected answer is correct. Our hypothesis is
that this behavior is due to the reasoning that modifies the probability of the
selected answer, as the LLM predicts the answer based on the input question and
the reasoning that supports the selection made. Therefore, LLM estimated
probabilities seem to have intrinsic limitations that should be understood in
order to use them in evaluation procedures. Interestingly, the same behavior
has been observed in humans, for whom explaining an answer increases confidence
in its correctness.
LLMs are more confident in their answers when reasoning is provided before the answer, regardless of correctness, due to modifications in probability estimation.
Large Language Models (LLMs) Multiple Choice Question (MCQ) tests few shots chain of thought LLM confidence LLM-estimated probability