aluation of responses as an instruction following task. This self-created
AIF preference data is used as a training set.
Our overall self-alignment procedure is an iterative one, which proceeds by building a series
of such models, with the aim that each improves over the last. Importantly, because the
model can both improve its generation ability, and it acts as its own reward model through
the same generation mechanism, this means the reward model itself can improve through
these iterations, deviating from