assigned rewards by that same model. The latter
is implemented via LLM-as-a-Judge prompting, which can also be seen as an instruction
following task. A preference dataset is built from the generated data, and the next iteration
of the model is trained via DPO.
In our experiments, we start with a Llama 2 70B [Touvron et al., 2023] seed model fine-tuned
on Open Assistant [Köpf et al., 2023], and then perform the above training scheme. We
find that not only does the instruction following performance improve fr