title: Diffusion World Model
summary: We introduce Diffusion World Model (DWM), a conditional diffusion model
  capable of predicting multistep future states and rewards concurrently. As
  opposed to traditional one-step dynamics models, DWM offers long-horizon
  predictions in a single forward pass, eliminating the need for recursive
  quires. We integrate DWM into model-based value estimation, where the
  short-term return is simulated by future trajectories sampled from DWM. In the
  context of offline reinforcement learning, DWM can be viewed as a conservative
  value regularization through generative modeling. Alternatively, it can be seen
  as a data source that enables offline Q-learning with synthetic data. Our
  experiments on the D4RL dataset confirm the robustness of DWM to long-horizon
  simulation. In terms of absolute performance, DWM significantly surpasses
  one-step dynamics models with a 44% performance gain, and achieves
  state-of-the-art performance.
paper_authors._id: 65c3062182d1feaf1b21a643
paper_authors.name: Zihan Ding
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-02-07T04:25:07.178Z
paper_authors._id: 627be1464d0858f00350ebba
paper_authors.avatarUrl: /avatars/d2ecacf46f416f46a0df29e9153b3f11.svg
paper_authors.fullname: Zihan Ding
paper_authors.type: user
paper_authors.user: quantumiracle
paper_authors._id: 65c3062182d1feaf1b21a644
paper_authors.name: Amy Zhang
paper_authors._id: 65c3062182d1feaf1b21a645
paper_authors.name: Yuandong Tian
paper_authors._id: 65c3062182d1feaf1b21a646
paper_authors.name: Qinqing Zheng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-07T09:59:36.169Z
paper_authors._id: 64d27579dafee18faf9308ac
paper_authors.avatarUrl: /avatars/8914a47244017c3541d3d5ac5b2d0372.svg
paper_authors.fullname: Qinqing Zheng
paper_authors.type: user
paper_authors.user: goodsleep
paper_title: Diffusion World Model
paper_summary: We introduce Diffusion World Model (DWM), a conditional diffusion model
  capable of predicting multistep future states and rewards concurrently. As
  opposed to traditional one-step dynamics models, DWM offers long-horizon
  predictions in a single forward pass, eliminating the need for recursive
  quires. We integrate DWM into model-based value estimation, where the
  short-term return is simulated by future trajectories sampled from DWM. In the
  context of offline reinforcement learning, DWM can be viewed as a conservative
  value regularization through generative modeling. Alternatively, it can be seen
  as a data source that enables offline Q-learning with synthetic data. Our
  experiments on the D4RL dataset confirm the robustness of DWM to long-horizon
  simulation. In terms of absolute performance, DWM significantly surpasses
  one-step dynamics models with a 44% performance gain, and achieves
  state-of-the-art performance.
paper_ai_summary: A conditional diffusion model (DWM) is introduced for long-horizon predictions in model-based reinforcement learning, surpassing traditional one-step models and achieving state-of-the-art performance.
paper_ai_keywords: Diffusion World Model conditional diffusion model long-horizon predictions single forward pass model-based value estimation offline reinforcement learning generative modeling synthetic data D4RL dataset one-step dynamics models