3 We omit WNLI as in Devlin et al. (2018) because the no
current algorithm beats the baseline of predicting the majority
class.

4 We treat MNLIm and MNLImm as separate tasks with individ-
ually tuned hyperparameters. However, they could be combined
into one model, leaving 8× overall.
Parameter-Efficient Transfer Learning for NLP