36.0
84.5
32.6
29.0
67.2
68.7
90.3
71.1
57.0
54.7
55.8
59.0
54.9
50.5
40.6
56.8

Table 9: Automatic evaluation results of LaMini-T5 language models and their baselines on 15 NLP tasks. “Aver-
age” indicates the micro-average of the individual task results.

Flan-T5

LaMini-Flan-T5
77M

Flan-T5

LaMini-Flan-T5
248M

Flan-T5

LaMini-Flan-T5
783M

# of params.
OpenBookQA
SciQ
RACE
ARC
PIQA
ReCoRD
SST
MRPC
RTE
MultiNLI
MultiNLI (mis)
WSC273
WinoGrande
WiC
HellaSwag
Average

27.0
89.0
29.7
22.3
61.9
57.7
87.3
63.2
60.3
42.4
42.5
53.1
50.0
51.3
29.1
51.1

30.0
79.4
28.9
24.0
61.9
53.8
85.7
58.6
56.3
53.2
53.2
54.6
50.1
50.8
28.6
51.3

33.0
86.2
34.4
27.3
65.7
61.3
92.2
74.8
66.1
66.6
66.8
60.4
53.0
60.8
34.6
58.9

31.2
93.8
40.9
30.7
72.2
76.7
94.0
82.6
87.4
72.4
72.0
66.7
59.9
64.7
48.7
66.3

34.0
86.7
32.8
31.8
70.6
70.4
93.1
77.9
65.0
61.4
61.0
64.1
56.0
63.8
43.7
60.8
LaMini-LM- A Diverse Herd of Distilled Models from Large-Scale Instructions