Paper Review, Deep Learning, Rlhf, Llm, NLP.
them hard to correct once the policy lacks diversity. It is difficult to balance efficiency and avoiding overfitting by the policy: In practice, the three key steps of RLHF are often performed serially, leading to off-distribution inaccuracies in the reward model, where the policy will then learn