title: TD-MPC2: Scalable, Robust World Models for Continuous Control
summary: TD-MPC is a model-based reinforcement learning (RL) algorithm that performs
  local trajectory optimization in the latent space of a learned implicit
  (decoder-free) world model. In this work, we present TD-MPC2: a series of
  improvements upon the TD-MPC algorithm. We demonstrate that TD-MPC2 improves
  significantly over baselines across 104 online RL tasks spanning 4 diverse task
  domains, achieving consistently strong results with a single set of
  hyperparameters. We further show that agent capabilities increase with model
  and data size, and successfully train a single 317M parameter agent to perform
  80 tasks across multiple task domains, embodiments, and action spaces. We
  conclude with an account of lessons, opportunities, and risks associated with
  large TD-MPC2 agents. Explore videos, models, data, code, and more at
  https://nicklashansen.github.io/td-mpc2
paper_authors._id: 6539ba7b48b006575d3cb566
paper_authors.name: Nicklas Hansen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-10-26T07:32:34.094Z
paper_authors._id: 62d8ab94bf50eca8389bbc01
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62d8ab94bf50eca8389bbc01/sXJupPzRO5ZV-BiQRdFeV.png
paper_authors.fullname: Nicklas Hansen
paper_authors.type: user
paper_authors.user: nicklashansen
paper_authors._id: 6539ba7b48b006575d3cb567
paper_authors.name: Hao Su
paper_authors._id: 6539ba7b48b006575d3cb568
paper_authors.name: Xiaolong Wang
paper_title: TD-MPC2: Scalable, Robust World Models for Continuous Control
paper_summary: TD-MPC is a model-based reinforcement learning (RL) algorithm that performs
  local trajectory optimization in the latent space of a learned implicit
  (decoder-free) world model. In this work, we present TD-MPC2: a series of
  improvements upon the TD-MPC algorithm. We demonstrate that TD-MPC2 improves
  significantly over baselines across 104 online RL tasks spanning 4 diverse task
  domains, achieving consistently strong results with a single set of
  hyperparameters. We further show that agent capabilities increase with model
  and data size, and successfully train a single 317M parameter agent to perform
  80 tasks across multiple task domains, embodiments, and action spaces. We
  conclude with an account of lessons, opportunities, and risks associated with
  large TD-MPC2 agents. Explore videos, models, data, code, and more at
  https://nicklashansen.github.io/td-mpc2
paper_ai_summary: TD-MPC2, an improved model-based reinforcement learning algorithm, achieves strong performance across various RL tasks with consistent results using a single set of hyperparameters and benefits from larger models and datasets.
paper_ai_keywords: model-based reinforcement learning RL TD-MPC TD-MPC2 local trajectory optimization latent space implicit world model decoder-free hyperparameters