• Rate-D: The generated text is completely un-

rors in content.

acceptable.

More details about the human evaluation protocol
are presented in Appendix C.

We randomly sample 20 examples from each
subset of DDDALL and one of the co-authors scores
the generated text. In general, both the generated
instructions and the generated responses are of high
quality as shown in Figure 4. During the annota-

tion process, we observe that examples from (cid:98)XXXP3
and (cid:98)XXXFLAN are much shorter than those from XXXP3

and XXXFLAN and their associated context are sig-
niﬁcantly shorter and easier, which conﬁrms our
observation in Table 1. Another noteworthy obser-
vation is that gpt-3.5-turbo is even more prone
to generated the responses with factual errors when
we provide the topics.

4 Experiment
LaMini-LM- A Diverse Herd of Distilled Models from Large-Scale Instructions