there is a number of works that instruction-tune the
vanilla language models on the model-generated
instructions (Taori et al., 2023; Chiang et al., 2023;
Anand et al., 2023; Chen et al., 2023).
LaMini-LM- A Diverse Herd of Distilled Models from Large-Scale Instructions