PWM: Policy Learning with Large World Models
<|reference_start|>PWM: Policy Learning with Large World Models: Reinforcement Learning (RL) has achieved impressive results on complex tasks but struggles in multi-task settings with different embodiments. World models offer scalability by learning a simulation of the environment, yet they often rely on inefficient gradient-free optimization methods. We introduce Policy learning with large World Models (PWM), a novel model-based RL algorithm that learns continuous control policies from large multi-task world models. By pre-training the world model on offline data and using it for first-order gradient policy learning, PWM effectively solves tasks with up to 152 action dimensions and outperforms methods using ground-truth dynamics. Additionally, PWM scales to an 80-task setting, achieving up to 27% higher rewards than existing baselines without the need for expensive online planning. Visualizations and code available at https://www.imgeorgiev.com/pwm<|reference_end|>
@article{georgiev2024pwm:,
    title={PWM: Policy Learning with Large World Models},
    author={Ignat Georgiev, Varun Giridhar, Nicklas Hansen and Animesh Garg},
    journal={arXiv preprint arXiv:2407.02466},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.02466},
    primaryClass={cs.LG cs.AI cs.RO}
}