title: FastTD3: Simple, Fast, and Capable Reinforcement Learning for Humanoid
    Control
summary: Reinforcement learning (RL) has driven significant progress in robotics, but
  its complexity and long training times remain major bottlenecks. In this
  report, we introduce FastTD3, a simple, fast, and capable RL algorithm that
  significantly speeds up training for humanoid robots in popular suites such as
  HumanoidBench, IsaacLab, and MuJoCo Playground. Our recipe is remarkably
  simple: we train an off-policy TD3 agent with several modifications -- parallel
  simulation, large-batch updates, a distributional critic, and carefully tuned
  hyperparameters. FastTD3 solves a range of HumanoidBench tasks in under 3 hours
  on a single A100 GPU, while remaining stable during training. We also provide a
  lightweight and easy-to-use implementation of FastTD3 to accelerate RL research
  in robotics.
paper_authors._id: 6838a05e80df874a4f2360d8
paper_authors.name: Younggyo Seo
paper_authors._id: 6838a05e80df874a4f2360d9
paper_authors.name: Carmelo Sferrazza
paper_authors._id: 6838a05e80df874a4f2360da
paper_authors.name: Haoran Geng
paper_authors._id: 6838a05e80df874a4f2360db
paper_authors.name: Michal Nauman
paper_authors._id: 6838a05e80df874a4f2360dc
paper_authors.name: Zhao-Heng Yin
paper_authors._id: 6838a05e80df874a4f2360dd
paper_authors.name: Pieter Abbeel
paper_title: FastTD3: Simple, Fast, and Capable Reinforcement Learning for Humanoid
    Control
paper_summary: Reinforcement learning (RL) has driven significant progress in robotics, but
  its complexity and long training times remain major bottlenecks. In this
  report, we introduce FastTD3, a simple, fast, and capable RL algorithm that
  significantly speeds up training for humanoid robots in popular suites such as
  HumanoidBench, IsaacLab, and MuJoCo Playground. Our recipe is remarkably
  simple: we train an off-policy TD3 agent with several modifications -- parallel
  simulation, large-batch updates, a distributional critic, and carefully tuned
  hyperparameters. FastTD3 solves a range of HumanoidBench tasks in under 3 hours
  on a single A100 GPU, while remaining stable during training. We also provide a
  lightweight and easy-to-use implementation of FastTD3 to accelerate RL research
  in robotics.
paper_ai_summary: FastTD3, an enhanced RL algorithm with parallel simulation and distributional critic, significantly accelerates training for humanoid robots.
paper_ai_keywords: reinforcement learning RL FastTD3 off-policy TD3 parallel simulation large-batch updates distributional critic HumanoidBench IsaacLab MuJoCo Playground