ion-
following examples to add to its own training set.
These skills are used so that the model can perform self-alignment, i.e., they are the
components used to iteratively train itself using AI Feedback (AIF).
Self-instruction creation consists of generating candidate responses and then the model itself
judging their quality, i.e., it acts as its own reward model, replacing the need for an external
one. This is implemented via the LLM-as-a-Judge mechanism [Zheng et al., 2023b], i.e., by
formulating the ev