7 Conclusion
We introduce SELF-INSTRUCT, a task-agnostic
method to improve the instruction-following capa-
bilities of language models via its own generation
of instruction data (instruction, input, and output
samples) and bootstrapping with it. Our method
conducts instruction-tuning of the original model
on the pruned subset of generated samples. On
experimenting with vanilla GPT3, we observe a
33% absolute improvement over the original model
on SUPER-NATURALINSTRUCTIONS. This perfor-
mance is on par with InstructGPT001 , which is
trained with private user data and expensive hu-
man annotations. Furthermore, we curate a set of
expert-written instructions for novel tasks. Human
evaluation on this set shows that tuning GPT3 with
SELF-INSTRUCT outperforms using existing pub-
lic instruction datasets by a large margin, leaving
only a 5% absolute gap behind InstructGPT001. We
hope SELF-INSTRUCT can serve as the ﬁrst step to
align pretrained language models to follow human
SELF-INSTRUCT- Aligning Language Model with Self Generated Instructions