Previous Best (↑)

Zero-Shot One-Shot

Fine-Tune

Ours (↑)
Fine-Tune

Metric
F1
acc
avg
acc
acc
acc
acc
F1
acc
acc
acc
ROUGE-2
ROUGE-2

Name
SQuADv2
SQuADv2
SuperGLUE
BoolQ
Copa
RTE
WiC
MultiRC
WSC
ReCoRD
CB
XSum
CNN-DM
WinoGrande XL acc
acc
ANLI R3
acc
ARC-Easy
ARC-Challenge
acc
em
CB TriviaQA
em
CB NatQA
CB WebQA
em

Split
dev
dev
test

dev/test
dev/test
dev/test
dev/test
dev/test
dev/test
dev/test
dev/test

test
test
dev
test
test
test
dev
test
test

70.0e
64.6e

–

82.8e
92.0e
71.5e
52.7e
72.9d
83.9e
90.8e
73.2e

–
–

73.2d
40.8e
76.6e
53.2
74.8e
23.9e
25.3

96.2a
91.3a
90.9
92.0
98.2
94.1
77.9
88.6
97.3
96.4
99.2
24.6h
21.6a

–

53.4
92.7g
81.4g
61.6b
41.5c
42.8b

96.3
90.8
91.2
92.4
99.2
93.5
77.7
89.6
96.6
95.1
98.0
27.1
21.7
96.1
74.7
95.2
86.5
62.3
41.9
47.4
ST-MOE- DESIGNING STABLE AND TRANSFERABLE SPARSE EXPERT MODELS