D.3 Additional Robustness Analysis

As the experiments in the main paper use a ﬁxed number of few-shot exemplars (8; as constrained by
the input length of 1024 tokens), we verify that the chain-of-thought prompting is robust to various
numbers of few-shot exemplars. We run experiments for LaMDA 137B, comparing chain-of-thought
prompting with standard prompting for the ﬁve datasets where standard prompting had a mostly ﬂat
scaling curve (the largest model did not achieve high performance). As shown in Figure 11, the
improvement of chain-of-thought prompting over standard prompting remains robust to varying the
number of few-shot exemplars in the prompt.

28

Standard prompting

Chain of thought prompting

GSM8K

MultiArith
(MAWPS)

Sports

Understanding

Coin Flip

Last Letter

Concatenation

)

%

(

e
t
a
r

e
v
l
o
S

15

10

5

0

60

40

20

0

100

75

50

25

0

100

75

50

25

0

100

75

50

25

0

1 2 4 6 8

1 2 4 6 8

1 2 4 6 8

1 2 4 6 8

1

2

3

4
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models