Paper Review, Deep Learning, Rlhf, Llm, NLP.
facilitates even more potent exploits. Policy Misgeneralization Policies can perform poorly in deployment even if rewards seen during training were perfectly correct: Even if the rewards during training are accurate, policies can perform poorly during deployment because the deployment environment