then conducting instruction tuning with this gener-
ated data in order to align the language model to
follow instructions better. This pipeline is depicted
in Figure 1.
3.1 Deﬁning Instruction Data
The instruction data we want to generate contains
a set of instructions {𝐼𝑡}, each of which deﬁnes a
task 𝑡 in natural language. Each task has one or
more input-output instances (𝑋𝑡, 𝑌𝑡). A model 𝑀
is expected to produce the output 𝑦, given the task
instruction 𝐼𝑡 and the instance input 𝑥: 𝑀(𝐼𝑡, 𝑥) =
𝑦, for (𝑥, 𝑦) ∈ (𝑋𝑡, 𝑌𝑡). Note that the instruction
and instance input does not have a strict boundary
in many cases. For example, “write an essay about
school safety” can be a valid instruction that we
expect models to respond to directly, while it can
also be formulated as “write an essay about the fol-
lowing topic” as the instruction, and “school safety”
as an instance input. To encourage the diversity of
the data format, we allow such instructions that do
SELF-INSTRUCT- Aligning Language Model with Self Generated Instructions