on diverse open-source Reward Modeling datasets. We have not yet observed any such divergence, and
hypothesize that iterative model updates may be helping to prevent this.
As a last verification step to ensure no regression between our new model and the previous one, we use both
to sample during the next annotation iteration. This enables a model comparison “for free” on new prompts
and can help to increase diversity when sampling.
Llama2