9.7

100.0 100.0 68.6 70.9 63.2 57.9 31.0 41.0 72.7 60.6 61.8 61.8 51.4 57.1 45.2 29.0 40.0 26.5 64.5 58.1
90.9
81.4 76.7 65.8 60.5 22.0 38.0 72.7 60.6 67.6 67.6 51.4 57.1 35.5 32.3 45.3 32.4 61.3 71.0

90.9

100.0 100.0 75.6 86.0 73.7 57.9 53.0 55.0 69.7 57.6 85.3 76.5 74.3 68.6 51.6 51.6 53.5 41.8 83.9 64.5
90.9 100.0 83.7 84.9 76.3 71.1 54.0 71.0 87.9 75.8 79.4 79.4 82.9 77.1 64.5 61.3 60.6 54.7 90.3 77.4
25.6 17.4
45.5
0.0
36.4
41.9 47.7 36.8 34.2 32.0 33.0 48.5 27.3 38.2 29.4 40.0 31.4 19.4 32.3 26.5 17.1 29.0 38.7

30.3 27.3 29.4

11.4 28.6 19.4

18.2
45.5

24.0

35.5

24.1

7.9

0.0

0.0

2.6

5.0

8.8

780M SwitchLARGE

0.0
FLAN-SwitchLARGE 54.5

9.1
54.5

27.9 24.4 26.3 21.1 22.0 20.0 21.2 21.2 29.4 11.8 48.6 22.9 32.3 32.3 27.6
16.1 19.4
53.5 59.3 47.4 28.9 24.0 23.0 60.6 30.3 41.2 35.3 42.9 60.0 38.7 25.8 36.5 25.3 51.6 38.7

4.1

11B SwitchXXL

FLAN-SwitchXXL
Mixture-of-Experts