2.2 EXPERIMENTAL SETUP

We considered P3 (Sanh et al., 2021), a publicly available multi-task suite that includes 62 NLP
datasets grouped into 12 task types (a full list of these datasets appears in the appendix). For each
dataset, the P3 suite includes various natural language prompt formats, referred to as templates, which
represent diverse natural manners of presenting and addressing the NLP query (e.g., in the case of a
natural language inference dataset, a template could be: “If {Premise} is true, is it
also true that {Hypothesis}?"). We also leveraged T0, a model based on T5 that was
ﬁne tuned on the P3 training set by the same authors. More speciﬁcally, they released three models,
dubbed T0, T0+, and T0++, which they ﬁne tuned on 39, 49, and 55 of the P3 tasks, respectively.1

1Sanh et al. (2021) focused on measuring generalization to unseen tasks and therefore they did not train on

all 62 datasets.

4

Preprint.
STANDING ON THE SHOULDERS OF GIANT FROZEN LANGUAGE MODELS