Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning
  by Large Language Models
Large language models (LLMs) have recently been shown to deliver impressive
performance in various NLP tasks. To tackle multi-step reasoning tasks,
few-shot chain-of-thought (CoT) prompting includes a few manually crafted
step-by-step reasoning demonstrations which enable LLMs to explicitly generate
reasoning steps and improve their reasoning task accuracy. To eliminate the
manual effort, Zero-shot-CoT concatenates the target problem statement with
"Let's think step by step" as an input prompt to LLMs. Despite the success of
Zero-shot-CoT, it still suffers from three pitfalls: calculation errors,
missing-step errors, and semantic misunderstanding errors. To address the
missing-step errors, we propose Plan-and-Solve (PS) Prompting. It consists of
two components: first, devising a plan to divide the entire task into smaller
subtasks, and then carrying out the subtasks according to the plan. To address
the calculation errors and improve the quality of generated reasoning steps, we
extend PS prompting with more detailed instructions and derive PS+ prompting.
We evaluate our proposed prompting strategy on ten datasets across three
reasoning problems. The experimental results over GPT-3 show that our proposed
zero-shot prompting consistently outperforms Zero-shot-CoT across all datasets
by a large margin, is comparable to or exceeds Zero-shot-Program-of-Thought
Prompting, and has comparable performance with 8-shot CoT prompting on the math
reasoning problem. The code can be found at
https://github.com/AGI-Edgerunners/Plan-and-Solve-Prompting.
6459d787130a95cb18fe52bc
Lei Wang
claimed_verified
2023-05-24T15:31:13.452Z
646def60df618b303b419323
/avatars/97aa761d5255abf230304cfeade87835.svg
Lei Wang
user
demolei
6459d787130a95cb18fe52bd
Wanyu Xu
6459d787130a95cb18fe52be
Yihuai Lan
6459d787130a95cb18fe52bf
Zhiqiang Hu
claimed_verified
2024-06-27T09:48:21.793Z
637f228152229c63921119c3
https://cdn-avatars.huggingface.co/v1/production/uploads/637f228152229c63921119c3/acwXorra1r9_7i3KlBFjS.jpeg
Zhiqiang Hu
user
Zhiqiang007
6459d787130a95cb18fe52c0
Yunshi Lan
admin_assigned
2023-05-24T10:57:00.694Z
640447573d49e1e066b3ee12
/avatars/9715081d88d130b27d29b050f5c733c7.svg
Yunshi Lan
user
YunshiECNU
6459d787130a95cb18fe52c1
Roy Ka-Wei Lee
6459d787130a95cb18fe52c2
Ee-Peng Lim
Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning
  by Large Language Models
Large language models (LLMs) have recently been shown to deliver impressive
performance in various NLP tasks. To tackle multi-step reasoning tasks,
few-shot chain-of-thought (CoT) prompting includes a few manually crafted
step-by-step reasoning demonstrations which enable LLMs to explicitly generate
reasoning steps and improve their reasoning task accuracy. To eliminate the
manual effort, Zero-shot-CoT concatenates the target problem statement with
"Let's think step by step" as an input prompt to LLMs. Despite the success of
Zero-shot-CoT, it still suffers from three pitfalls: calculation errors,
missing-step errors, and semantic misunderstanding errors. To address the
missing-step errors, we propose Plan-and-Solve (PS) Prompting. It consists of
two components: first, devising a plan to divide the entire task into smaller
subtasks, and then carrying out the subtasks according to the plan. To address
the calculation errors and improve the quality of generated reasoning steps, we
extend PS prompting with more detailed instructions and derive PS+ prompting.
We evaluate our proposed prompting strategy on ten datasets across three
reasoning problems. The experimental results over GPT-3 show that our proposed
zero-shot prompting consistently outperforms Zero-shot-CoT across all datasets
by a large margin, is comparable to or exceeds Zero-shot-Program-of-Thought
Prompting, and has comparable performance with 8-shot CoT prompting on the math
reasoning problem. The code can be found at
https://github.com/AGI-Edgerunners/Plan-and-Solve-Prompting.
A new prompting strategy, Plan-and-Solve (PS) and its extension PS+, improves the reasoning capabilities of LLMs by reducing errors in multi-step reasoning tasks, outperforming existing zero-shot methods.
large language models (LLMs) few-shot chain-of-thought (CoT) prompting zero-shot-CoT Plan-and-Solve (PS) Prompting PS+ prompting multi-step reasoning tasks calculation errors missing-step errors semantic misunderstanding errors subtasks GPT-3 zero-shot prompting Zero-shot-Program-of-Thought Prompting 8-shot CoT prompting