Paper Review, Deep Learning, Rlhf, Llm, NLP.
Data collection can introduce biases: Ideal feedback data should resemble the deployment distribution, especially highlighting difficult examples for the reward model. However, actual interactions with large language models (LLMs) might not align with any specific distribution causing potential