Paper Review, Deep Learning, Rlhf, Llm, NLP.
Evaluating Reward Models Evaluating reward models is difficult and expensive: If the true reward function is known, there are methods to assess the quality of the learned reward model. However, reward modeling is often used when the true reward isn’t known, preventing direct evaluation. Typically,