1500.0M 91.8

96.0

96.4

97.2

92.2

90.9

92.4

92.9

92.0

72.0

86.6

93.9

68.0

94.7

92.7

Table 2: RoBERTabase, RoBERTalarge, and DeBERTaXXL with different adaptation methods on the
GLUE benchmark. We report the overall (matched and mismatched) accuracy for MNLI, Matthew’s
correlation for CoLA, Pearson correlation for STS-B, and accuracy for other tasks. Higher is better
for all metrics. * indicates numbers published in prior works. † indicates runs conﬁgured in a setup
similar to Houlsby et al. (2019) for a fair comparison.
LORA