y introduced in Xu et al. [2023].
Starting from a seed model, as shown in Figure 1, in each iteration there is a process
arXiv:2401.10020v1  [cs.CL]  18 Jan 2024
Generate
responses
Generate
rewards
Preference 
pairs
DPO 
training
 select
Generated 
new prompts
Self-Instruction creation
Instruction following training
Next iteration model
Seed model
(for t=1)
Figure 1: Self-Rewarding Language Models. Our self-alignment method consists of two
steps: (i) Self-Instruction creation: newly created prompts are used