34.2 51.4
20.1 19.2
20.2 19.6
19.0 20.3
37.0 44.1
53.6 57.9
14.7 15.2
12.0 19.2
19.0 24.0
79.5 73.5
82.3 77.9
19.8 24.9
65.4 68.1
78.1 79.9

59.0 53.3
46.4 24.9
52.6 45.2
54.1 46.8
59.6 62.2
62.4 65.4
20.6
0.9
45.8 35.7
53.6 50.0
65.9 65.4
67.1 73.2
55.6 53.5
58.4 63.4
68.6 77.8

Table 5: Standard prompting versus chain of thought prompting enables length generalization to
longer inference examples on two symbolic manipulation tasks.

Last Letter Concatenation
2

OOD: 3

OOD: 4

Coin Flip (state tracking)

2

OOD: 3

OOD: 4

Model
UL2
LaMDA 420M

20B

2B
8B
68B
137B

PaLM 8B
62B
540B

standard CoT standard CoT standard CoT standard CoT standard CoT standard CoT
48.7 50.4
49.5 49.1
49.8 50.2
51.2 50.6
50.9 59.6
49.1 74.5
50.9 51.8
38.3 72.4
54.8 90.2

70.4 67.1
52.9 49.6
54.9 55.3
52.9 55.5
56.2 83.2
49.0 99.6
60.0 74.4
91.4 96.8
98.1 100.0

51.6 52.2
50.0 50.5
47.4 48.7
48.2 49.6
50.4 69.1
50.7 91.0
47.3 57.1
43.9 91.0
49.3 98.6
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models