T5LARGE
FLAN-T5LARGE

T5XL
FLAN-T5XL

T5XXL
FLAN-T5XXL

PaLM
FLAN-PaLM

PaLM
FLAN-PaLM

PaLM
FLAN-PaLM
SwitchBASE
FLAN-SwitchBASE

SwitchLARGE
FLAN-SwitchLARGE

SwitchXXL
FLAN-SwitchXXL
GSSMALL
FLAN-GSSMALL

GSBASE
FLAN-GSBASE

GSLARGE
FLAN-GSLARGE

GSXL
FLAN-GSXL
ECSMALL
FLAN-ECSMALL

ECBASE
FLAN-ECBASE

ECLARGE
FLAN-ECLARGE

ECXL
FLAN-ECXL
STBASE
FLAN-STBASE

ST32B
FLAN-ST32B

0.3G
0.3G

1.0G
1.0G

3.6G
3.6G

13.9G
13.9G

12.6G
12.6G

91.6G
91.6G

847G
847G
0.3G
0.3G

1.0G
1.0G

13.9G
13.9G
0.06G
0.06G

0.3G
0.3G

1.0G
1.0G

03.6G
3.6G
0.06G
0.06G

0.3G
0.3G

1.0G
1.0G

3.6G
3.6G
0.3G
0.3G

32.1G
32.1G

250M
250M

780M
780M

3B
3B

11B
11B

8B
8B

62B
62B

540B
540B
3.5B
3.5B

26B
26B

395B
395B
0.3B
0.3B

1.3B
1.3B

9.2B
9.2B

17.4B
17.4B
0.3B
0.3B

1.3B
1.3B

9.2B
9.2B

17.4B
17.4B
1.3B
1.3B

259B
259B

25.7
35.6

25.1
44.7

25.3
50.3

26.1
52.6

24.3
49.3

55.1
59.6

71.3
73.5
28.3
38.0

24.0
46.1

24.6
55.6
23.9
32.6

25.0
39.9

26.4
47.8

25.7
51.1
25.3
34.1

25.0
42.7
Mixture-of-Experts