3.3 Finetuning the LM to Follow Instructions
After the creation of the large-scale instruction data,
we use this data to ﬁnetune the original language
model (i.e., SELF-INSTRUCT). To do this, we con-
catenate the instruction and instance input as a
prompt and train the model to generate the instance
output in a standard supervised way. To make the
model robust to diﬀerent formats, we use multiple
templates to encode the instruction and instance
input together. For example, the instruction can
be preﬁxed with “Task:” or not, the input can be
preﬁxed with “Input:” or not, “Output:” can be
appended at the end of the prompt, and diﬀerent
numbers of break lines can be put in the middle,
etc.
4 SELF-INSTRUCT Data from GPT3
In this section, we apply our method for inducing
instructiondatatoGPT3 asacasestudy. Weusethe
largest GPT3 language model (“davinci” engine)
accessedthroughtheOpenAIAPI5.Theparameters
for making queries are described in Appendix A.1.
SELF-INSTRUCT- Aligning Language Model with Self Generated Instructions