Model
-
-
-
-
80M T5-Small

250M T5-Base

780M T5-Large

2.7

0.0

51.3

Flan-T5-Base

Flan-T5-Small

Direct CoT Direct CoT Direct
54.0 69.2 57.8 48.1 37.6
davinci
text-davinci-002
90.0 87.6 57.8 56.1 55.6
90.0 90.8 63.6 63.6 58.8
text-davinci-003
code-davinci-002 88.4 92.8 63.6 54.0 63.6
40.0
20.0
54.0 39.6 48.1 42.8 22.4
46.0 45.6 51.9 38.0 20.0
44.8 46.8 48.1 45.5 22.8
46.0 49.2 51.9 26.2 20.8
55.6 56.4 57.8 56.1 21.6
Flan-T5-Large
55.2 47.2 52.4 26.7 21.6
T5-XL
56.0 44.0 63.6 56.7 44.4
Flan-T5-XL
49.6 65.2 52.4
35.2
T5-XXL
54.4 62.4 60.4 55.6 54.0
Flan-T5-XXL
58.4 66.0 48.1 43.3 16.4
PaLM
48.8 52.8 60.4 54.0 10.8
Flan-PaLM
69.2 70.8 59.4 54.5 39.2
PaLM
66.8 73.6 64.2 62.6 42.8
Flan-PaLM
cont-PaLM
78.4 84.8 61.5 58.8 50.4
Flan-cont-PaLM 76.8 85.6 66.3 64.2 52.0
83.2 80.0 61.0 59.4 53.6
86.0 83.2 65.2 63.1 58.0
82.0 70.8 64.2 62.0 55.6
86.0 86.4 63.6 65.8 61.2

Flan-U-PaLM

Flan-PaLM

1.6

540B PaLM

540B U-PaLM

3B

11B

8B

62B

62B
Scaling Instruction-Finetuned Language Models