BERTBASE
Adapters
91.7 ± 0.2
84.5 ± 0.2
92.9 ± 0.3
84.1 ± 0.2
82.5 ± 0.3
38.7 ± 0.1
82.7 ± 0.3
79.0 ± 0.5
75.9 ± 0.3
44.1 ± 0.2
33.9 ± 1.4
71.7 ± 1.1
60.6 ± 1.4
77.3 ± 0.1
55.4 ± 0.1
96.2 ± 0.0
95.1 ± 2.2
73.3
1.19×
1.14%

91.1
84.5
91.9
84.9
81.1
36.3
82.7
81.0
76.8
43.8
33.5
70.6
54.3
75.6
54.5
95.2
98.5

72.7
—
—

Table 2. Test accuracy for additional classiﬁcation tasks. In these experiments we transfer from the BERTBASE model. For each task
and algorithm, the model with the best validation set accuracy is chosen. We report the mean test accuracy and s.e.m. across runs with
different random seeds.
Parameter-Efficient Transfer Learning for NLP