Paper Review, Deep Learning, Rlhf, Llm, NLP.
example of this bias is the learned correlation between sounding confident and being correct, reinforcing confidence as a desirable trait in the policy. RL contributes to mode collapse: RL fine-tuning reduces the diversity of samples produced by a model, leading to “mode collapse”. Studies have