-
-
-
-

davinci
text-davinci-002
text-davinci-003
code-davinci-002

54.0 69.2 57.8 48.1 37.6
90.0 87.6 57.8 56.1 55.6
90.0 90.8 63.6 63.6 58.8
88.4 92.8 63.6 54.0 63.6

80M T5-Small

Flan-T5-Small

0.0

40.0
20.0
54.0 39.6 48.1 42.8 22.4

51.3

2.7

250M T5-Base

Flan-T5-Base

46.0 45.6 51.9 38.0 20.0
48.4 46.4 52.4 47.1 18.0

780M T5-Large

Flan-T5-Large

46.0 49.2 51.9 26.2 20.8
64.0 58.0 56.1 20.9 24.4

3B

T5-XL
Flan-T5-XL

55.2 47.2 52.4 26.7 21.6
52.4 56.0 62.0 56.1 46.8

11B T5-XXL

Flan-T5-XXL

49.6 65.2 52.4
35.2
56.8 60.8 60.4 53.5 69.6

1.6

Flan-PaLM

8B
62B PaLM

Flan-PaLM

540B PaLM

Flan-PaLM
250M SwitchBASE

FLAN-SwitchBASE

48.8 52.8 60.4 54.0 10.8
69.2 70.8 59.4 54.5 39.2
66.8 73.6 64.2 62.6 42.8

83.2 80.0 61.0 59.4 53.6
86.0 83.2 65.2 63.1 58.0
0.0
0.0
51.2 42.8 55.1 55.6 18.8

10.7

2.7

0.0

780M SwitchLARGE

0.0
FLAN-SwitchLARGE 54.0 22.0 56.7 50.8 25.2

26.0

5.3

5.3

0.0

11B SwitchXXL

FLAN-SwitchXXL
Mixture-of-Experts