Paper Review, Deep Learning, Rlhf, Llm, NLP.
manifestation of this issue is LLMs displaying sycophantic behavior. Distributional Challenges The pretrained model introduces biases into policy optimization: RLHF in LLMs often starts with a base model pretrained on internet text, influencing the RL policy network. This approach has been