title: SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step
    Reasoning
summary: The recent progress in large language models (LLMs), especially the invention
  of chain-of-thoughts (CoT) prompting, makes it possible to solve reasoning
  problems. However, even the strongest LLMs are still struggling with more
  complicated problems that require non-linear thinking and multi-step reasoning.
  In this work, we explore whether LLMs have the ability to recognize their own
  errors, without resorting to external resources. In particular, we investigate
  whether they can be used to identify individual errors within a step-by-step
  reasoning. To this end, we propose a zero-shot verification scheme to recognize
  such errors. We then use this verification scheme to improve question-answering
  performance, by using it to perform weighted voting on different generated
  answers. We test the method on three math datasets-GSM8K, MathQA, and MATH-and
  find that it successfully recognizes errors and, in turn, increases final
  predictive performance.
paper_authors._id: 64c9bc560986bd6fa25a42fe
paper_authors.name: Ning Miao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-08-02T08:06:08.373Z
paper_authors._id: 6347fa30e3f2de18545d894c
paper_authors.avatarUrl: /avatars/b977fbfe87d2d48e43ab0431ca336b1f.svg
paper_authors.fullname: NingMiao
paper_authors.type: user
paper_authors.user: NingMiao
paper_authors._id: 64c9bc560986bd6fa25a42ff
paper_authors.name: Yee Whye Teh
paper_authors._id: 64c9bc560986bd6fa25a4300
paper_authors.name: Tom Rainforth
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-08-02T12:28:11.781Z
paper_authors._id: 64ca27e3a81988d07319e842
paper_authors.avatarUrl: /avatars/e0c87517352c8e68ffe7ba8e6acc7999.svg
paper_authors.fullname: Tom Rainforth 
paper_authors.type: user
paper_authors.user: TomRainforth
paper_title: SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step
    Reasoning
paper_summary: The recent progress in large language models (LLMs), especially the invention
  of chain-of-thoughts (CoT) prompting, makes it possible to solve reasoning
  problems. However, even the strongest LLMs are still struggling with more
  complicated problems that require non-linear thinking and multi-step reasoning.
  In this work, we explore whether LLMs have the ability to recognize their own
  errors, without resorting to external resources. In particular, we investigate
  whether they can be used to identify individual errors within a step-by-step
  reasoning. To this end, we propose a zero-shot verification scheme to recognize
  such errors. We then use this verification scheme to improve question-answering
  performance, by using it to perform weighted voting on different generated
  answers. We test the method on three math datasets-GSM8K, MathQA, and MATH-and
  find that it successfully recognizes errors and, in turn, increases final
  predictive performance.
paper_ai_summary: A zero-shot verification scheme improves LLMs' question-answering performance by identifying and correcting errors in step-by-step reasoning.
paper_ai_keywords: large language models (LLMs) chain-of-thoughts (CoT) reasoning problems non-linear thinking multi-step reasoning zero-shot verification scheme weighted voting question-answering GSM8K MathQA MATH