B Response Generation

The Python code used to generate the response
can be found in Figure Figure 8. Before ask-
ing gpt-3.5-turbo to generate responses, we
ﬁrstly send a message as the “system” that re-
quires gpt-3.5-turbo to respond the instructions
as concise as possible to avoid the overly lengthy
responses.

C Human Evaluation Protocol
We present the human evaluation protocol as well
as the corresponding example for each rating level
in Table 7.

D Automatic Evaluation Datasets
We present the details of 15 downstream NLP tasks,
including the number of test examples and the cor-
responding evaluation metrics, in Table 8.

E Automatic Evaluation Results
The breakdown results given by LaMini-
T5, LaMini-Flan-T5, LaMini-Neo, LaMini-
Cerebras and LaMini-GPT are presented in
Table 9,Table 10,Table 11,Table 12 and Table 13
respectively.
LaMini-LM- A Diverse Herd of Distilled Models from Large-Scale Instructions