l. [2023].
2. Generate candidate responses: We then generate N diverse candidate responses
{y1
i , . . . , yN
i } for the given prompt xi from our model using sampling.
3. Evaluate candidate responses: Finally, we use the LLM-as-a-Judge ability of our
same model to evaluate its own candidate responses with scores rn
i ∈[0, 5] (see
Figure 2).
2.3
Instruction Following Training
As previously described, training is initially performed with the seed IFT and EFT data
(Section 2.1). This is then augmented with ad