Paper Review, Deep Learning, Rlhf, Llm, NLP.
over-optimization of rewards during training is challenging. Potential solutions might include measuring KL-shift or monitoring disagreement within a reward model ensemble. Incorporating RLHF into a Broader Framework for Safer AI Addressing Challenges with RLHF Addressing Challenges with Human