title: Deductive Verification of Chain-of-Thought Reasoning
summary: Large Language Models (LLMs) significantly benefit from Chain-of-Thought
  (CoT) prompting in performing various reasoning tasks. While CoT allows models
  to produce more comprehensive reasoning processes, its emphasis on intermediate
  reasoning steps can inadvertently introduce hallucinations and accumulated
  errors, thereby limiting models' ability to solve complex reasoning tasks.
  Inspired by how humans engage in careful and meticulous deductive logical
  reasoning processes to solve tasks, we seek to enable language models to
  perform explicit and rigorous deductive reasoning, and also ensure the
  trustworthiness of their reasoning process through self-verification. However,
  directly verifying the validity of an entire deductive reasoning process is
  challenging, even with advanced models like ChatGPT. In light of this, we
  propose to decompose a reasoning verification process into a series of
  step-by-step subprocesses, each only receiving their necessary context and
  premises. To facilitate this procedure, we propose Natural Program, a natural
  language-based deductive reasoning format. Our approach enables models to
  generate precise reasoning steps where subsequent steps are more rigorously
  grounded on prior steps. It also empowers language models to carry out
  reasoning self-verification in a step-by-step manner. By integrating this
  verification process into each deductive reasoning stage, we significantly
  enhance the rigor and trustfulness of generated reasoning steps. Along this
  process, we also improve the answer correctness on complex reasoning tasks.
  Code will be released at https://github.com/lz1oceani/verify_cot.
paper_authors._id: 647fda5f9466a3c9ba4eb353
paper_authors.name: Zhan Ling
paper_authors._id: 647fda5f9466a3c9ba4eb354
paper_authors.name: Yunhao Fang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-14T12:38:05.792Z
paper_authors._id: 640d68938aee167ccda391da
paper_authors.avatarUrl: /avatars/8e3fbf6ca10fe4e9b04a3a84d4e3e255.svg
paper_authors.fullname: Yunhao Fang
paper_authors.type: user
paper_authors.user: Seerkfang
paper_authors._id: 647fda5f9466a3c9ba4eb355
paper_authors.name: Xuanlin Li
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-07T10:09:56.679Z
paper_authors._id: 64139dbfb0c7659b8fd54eea
paper_authors.avatarUrl: /avatars/41b552870c7f5159c11135fa4671af35.svg
paper_authors.fullname: Xuanlin Li
paper_authors.type: user
paper_authors.user: xuanlinli17
paper_authors._id: 647fda5f9466a3c9ba4eb356
paper_authors.name: Zhiao Huang
paper_authors._id: 647fda5f9466a3c9ba4eb357
paper_authors.name: Mingu Lee
paper_authors._id: 647fda5f9466a3c9ba4eb358
paper_authors.name: Roland Memisevic
paper_authors._id: 647fda5f9466a3c9ba4eb359
paper_authors.name: Hao Su
paper_title: Deductive Verification of Chain-of-Thought Reasoning
paper_summary: Large Language Models (LLMs) significantly benefit from Chain-of-Thought
  (CoT) prompting in performing various reasoning tasks. While CoT allows models
  to produce more comprehensive reasoning processes, its emphasis on intermediate
  reasoning steps can inadvertently introduce hallucinations and accumulated
  errors, thereby limiting models' ability to solve complex reasoning tasks.
  Inspired by how humans engage in careful and meticulous deductive logical
  reasoning processes to solve tasks, we seek to enable language models to
  perform explicit and rigorous deductive reasoning, and also ensure the
  trustworthiness of their reasoning process through self-verification. However,
  directly verifying the validity of an entire deductive reasoning process is
  challenging, even with advanced models like ChatGPT. In light of this, we
  propose to decompose a reasoning verification process into a series of
  step-by-step subprocesses, each only receiving their necessary context and
  premises. To facilitate this procedure, we propose Natural Program, a natural
  language-based deductive reasoning format. Our approach enables models to
  generate precise reasoning steps where subsequent steps are more rigorously
  grounded on prior steps. It also empowers language models to carry out
  reasoning self-verification in a step-by-step manner. By integrating this
  verification process into each deductive reasoning stage, we significantly
  enhance the rigor and trustfulness of generated reasoning steps. Along this
  process, we also improve the answer correctness on complex reasoning tasks.
  Code will be released at https://github.com/lz1oceani/verify_cot.
paper_ai_keywords: Chain-of-Thought (CoT) deductive reasoning self-verification Natural Program step-by-step verification reasoning subprocesses deductive reasoning process answer correctness