with private user data and human annotations.
For further evaluation, we curate a set of
expert-written instructions for novel tasks, and
show through human evaluation that tuning
GPT3withSELF-INSTRUCT outperformsusing
existing public instruction datasets by a large
margin, leaving only a 5% absolute gap behind
InstructGPT001. SELF-INSTRUCT provides an
almostannotation-freemethodforaligningpre-
trained language models with instructions, and
we release our large synthetic dataset to facili-
tate future studies on instruction tuning2.
Introduction
SELF-INSTRUCT- Aligning Language Model with Self Generated Instructions