2023; Liang et al., 2023; Chen et al., 2023a). Du et al. (2023) implement a multi-agent debate
method by leveraging multiple instances of a single ChatGPT model and demonstrate significant
improvements on reasoning tasks. We adopt their method to test performance on GSM8K. For an
unbiased implementation, we use the exact same prompt as Du et al. (2023) and replicate their ex-
periment with the gpt-3.5-turbo-0301 model, incorporating 3 agents and 2 rounds of debate.
The only distinction is that, to reduce result variance, we test on the complete test set of GSM8K,
compared to their usage of 100 examples. For reference, we also report the results of self-consistency
(Wang et al., 2022), which prompts models to generate multiple responses and performs majority
voting to select the final answer.
Table 4 presents the results5.
The
results indicate that both multi-agent
debate and self-consistency achieve
significant
improvements over stan-
dard prompting.
However, when
LARGELANGUAGEMODELSCANNOTSELF-CORRECT REASONINGYET