31.2 25.0 26.8 19.5 42.9

37.5 25.0 39.0 17.1 42.9

18.2 19.2 23.1 41.7

30.0 10.0 31.2

20.0 10.0 15.6

Flan-U-PaLM

Flan-PaLM

40.9

27.3

31.8

20.0

0.0

0.0

9.1

6.2

0.0

7.1

0.0

0.0

9.1

0.0

0.0

0.0

9.4

0.0

540B PaLM

540B U-PaLM

3B

11B

8B

62B

62B

36

Table 13: MMLU[20:30] individual task performance.

MMLU

High School
Comp. Sci.

High School

European History

High School
Geography

High School
Gvmt & Politics

High School

Macroeconomics

High School

Math

High School

Microeconomics

High School

Physics

High School
Psychology

High School
Statistics

Model
-
-
-
-
80M T5-Small

250M T5-Base

780M T5-Large

0.0
0.0
0.0

Flan-T5-Base

Flan-T5-Small
Scaling Instruction-Finetuned Language Models