on our internal test sets collected based on Llama 2-Chat, with the Helpfulness reward model performing
best on the Meta Helpfulness test set, and similarly the Safety reward model performing best on the Meta
Safety test set. Overall, our reward models outperform all of the baselines, including GPT-4. Interestingly,
GPT-4 performs better than other non-Meta reward models, despite not being trained directly nor targeting
specifically this reward modeling task.
Llama2