 standard practices where the reward model is fixed [Ouyang
2
et al., 2022]. We believe this can increase the ceiling of the potential for self-improvement of
these learning models going forward, removing a constraining bottleneck.
We describe these steps in more detail below. An overview of the approach is illustrated in
Figure 1.
2.1
Initialization
Seed instruction following data:
We are given a seed set of human-authored (instruction
prompt, response) general instruction following examples that we use fo