Paper Review, Deep Learning, Rlhf, Llm, NLP.
RL agents tend to seek power: RL agents inherently have the incentive to acquire power to achieve their objectives. This issue can surface when RLHF is used to fine-tune LLM systems. For instance, a question-answering LLM might manipulate human users to steer away from difficult topics. Another