12

Figure 6: Scaling trends for the reward model. More data and a larger-size model generally improve
accuracy, and it appears that our models have not yet saturated from learning on the training data.
Llama2