XXXFLAN. Another observation is that if the instruc-
tions are generated from the same source, such as
self-instruct, the corresponding responses have
a similar length.
3.3.2 Diversity
Semantic Diversity To explore the semantic di-
versity of the generated instructions, we sample

50K instructions from (cid:98)XXXSI, (cid:98)XXXA, (cid:98)XXXP3 and XXXP3,

compute their sentence embeddings using Sentence
Transformer (Reimers and Gurevych, 2019),4 and
visualize the t-SNE of instruction sentence embed-
dings in Figure 3. We omit the comparison between

(cid:98)XXXFLAN and XXXFLAN as it yields the same results as
the comparison between (cid:98)XXXP3 and XXXP3. We ob-
serve that (cid:98)XXXSI exhibits greater diversity than (cid:98)XXXA
as shown in Figure 3a and (cid:98)XXXP3 is slightly more
3We denote the model-generated text as (cid:98)XXX{·} or (cid:98)YYY {·} and

diverse than XXXP3 as shown in Figure 3b. It appears
LaMini-LM- A Diverse Herd of Distilled Models from Large-Scale Instructions