19

B All Experimental Results

This section contains tables for experimental results for varying models and model sizes, on all
benchmarks, for standard prompting vs. chain-of-thought prompting.
For the arithmetic reasoning benchmarks, some chains of thought (along with the equations produced)
were correct, except the model performed an arithmetic operation incorrectly. A similar observation
was made in Cobbe et al. (2021). Hence, we can further add a Python program as an external
calculator (using the Python eval function) to all the equations in the generated chain of thought.
When there are multiple equations in a chain of thought, we propagate the external calculator results
from one equation to the following equations via string matching. As shown in Table 1, we see that
adding a calculator signiﬁcantly boosts performance of chain-of-thought prompting on most tasks.
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models