TD-MPC2: Scalable, Robust World Models for Continuous Control
TD-MPC is a model-based reinforcement learning (RL) algorithm that performs
local trajectory optimization in the latent space of a learned implicit
(decoder-free) world model. In this work, we present TD-MPC2: a series of
improvements upon the TD-MPC algorithm. We demonstrate that TD-MPC2 improves
significantly over baselines across 104 online RL tasks spanning 4 diverse task
domains, achieving consistently strong results with a single set of
hyperparameters. We further show that agent capabilities increase with model
and data size, and successfully train a single 317M parameter agent to perform
80 tasks across multiple task domains, embodiments, and action spaces. We
conclude with an account of lessons, opportunities, and risks associated with
large TD-MPC2 agents. Explore videos, models, data, code, and more at
https://nicklashansen.github.io/td-mpc2
6539ba7b48b006575d3cb566
Nicklas Hansen
claimed_verified
2023-10-26T07:32:34.094Z
62d8ab94bf50eca8389bbc01
https://cdn-avatars.huggingface.co/v1/production/uploads/62d8ab94bf50eca8389bbc01/sXJupPzRO5ZV-BiQRdFeV.png
Nicklas Hansen
user
nicklashansen
6539ba7b48b006575d3cb567
Hao Su
6539ba7b48b006575d3cb568
Xiaolong Wang
TD-MPC2: Scalable, Robust World Models for Continuous Control
TD-MPC is a model-based reinforcement learning (RL) algorithm that performs
local trajectory optimization in the latent space of a learned implicit
(decoder-free) world model. In this work, we present TD-MPC2: a series of
improvements upon the TD-MPC algorithm. We demonstrate that TD-MPC2 improves
significantly over baselines across 104 online RL tasks spanning 4 diverse task
domains, achieving consistently strong results with a single set of
hyperparameters. We further show that agent capabilities increase with model
and data size, and successfully train a single 317M parameter agent to perform
80 tasks across multiple task domains, embodiments, and action spaces. We
conclude with an account of lessons, opportunities, and risks associated with
large TD-MPC2 agents. Explore videos, models, data, code, and more at
https://nicklashansen.github.io/td-mpc2
TD-MPC2, an improved model-based reinforcement learning algorithm, achieves strong performance across various RL tasks with consistent results using a single set of hyperparameters and benefits from larger models and datasets.
model-based reinforcement learning RL TD-MPC TD-MPC2 local trajectory optimization latent space implicit world model decoder-free hyperparameters