title: Diffusion for World Modeling: Visual Details Matter in Atari
summary: World models constitute a promising approach for training reinforcement
  learning agents in a safe and sample-efficient manner. Recent world models
  predominantly operate on sequences of discrete latent variables to model
  environment dynamics. However, this compression into a compact discrete
  representation may ignore visual details that are important for reinforcement
  learning. Concurrently, diffusion models have become a dominant approach for
  image generation, challenging well-established methods modeling discrete
  latents. Motivated by this paradigm shift, we introduce DIAMOND (DIffusion As a
  Model Of eNvironment Dreams), a reinforcement learning agent trained in a
  diffusion world model. We analyze the key design choices that are required to
  make diffusion suitable for world modeling, and demonstrate how improved visual
  details can lead to improved agent performance. DIAMOND achieves a mean human
  normalized score of 1.46 on the competitive Atari 100k benchmark; a new best
  for agents trained entirely within a world model. To foster future research on
  diffusion for world modeling, we release our code, agents and playable world
  models at https://github.com/eloialonso/diamond.
paper_authors._id: 664d594b9299555621c06496
paper_authors.name: Eloi Alonso
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-22T09:12:34.680Z
paper_authors._id: 66476d39156169f872095c0c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/ja4hmXYNBcprZau6Miux1.png
paper_authors.fullname: Eloi Alonso
paper_authors.type: user
paper_authors.user: eloialonso
paper_authors._id: 664d594b9299555621c06497
paper_authors.name: Adam Jelley
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-22T09:12:52.406Z
paper_authors._id: 66476cf764454e8f1c792345
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/66476cf764454e8f1c792345/MLviMsGieflLURaFxGbUP.jpeg
paper_authors.fullname: Adam Jelley
paper_authors.type: user
paper_authors.user: AdamJelley
paper_authors._id: 664d594b9299555621c06498
paper_authors.name: Vincent Micheli
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-22T09:13:28.538Z
paper_authors._id: 5f0d599719cb630495b81524
paper_authors.avatarUrl: /avatars/3c7472d1644d0c78331fafa88d16f0a6.svg
paper_authors.fullname: Vincent Micheli
paper_authors.type: user
paper_authors.user: vincem
paper_authors._id: 664d594b9299555621c06499
paper_authors.name: Anssi Kanervisto
paper_authors._id: 664d594b9299555621c0649a
paper_authors.name: Amos Storkey
paper_authors._id: 664d594b9299555621c0649b
paper_authors.name: Tim Pearce
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-22T09:13:41.205Z
paper_authors._id: 636f5c246cd69d9a36ff8791
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/636f5c246cd69d9a36ff8791/CkqAKquinHhFsVu7qxBJm.jpeg
paper_authors.fullname: Tim Pearce
paper_authors.type: user
paper_authors.user: TeaPearce
paper_authors._id: 664d594b9299555621c0649c
paper_authors.name: François Fleuret
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-22T09:13:52.895Z
paper_authors._id: 6318c37e69ef5467e18fc63f
paper_authors.avatarUrl: /avatars/7f1ac593608a89926f4107b4c3dd91d4.svg
paper_authors.fullname: François Fleuret
paper_authors.type: user
paper_authors.user: francoisfleuret
paper_title: Diffusion for World Modeling: Visual Details Matter in Atari
paper_summary: World models constitute a promising approach for training reinforcement
  learning agents in a safe and sample-efficient manner. Recent world models
  predominantly operate on sequences of discrete latent variables to model
  environment dynamics. However, this compression into a compact discrete
  representation may ignore visual details that are important for reinforcement
  learning. Concurrently, diffusion models have become a dominant approach for
  image generation, challenging well-established methods modeling discrete
  latents. Motivated by this paradigm shift, we introduce DIAMOND (DIffusion As a
  Model Of eNvironment Dreams), a reinforcement learning agent trained in a
  diffusion world model. We analyze the key design choices that are required to
  make diffusion suitable for world modeling, and demonstrate how improved visual
  details can lead to improved agent performance. DIAMOND achieves a mean human
  normalized score of 1.46 on the competitive Atari 100k benchmark; a new best
  for agents trained entirely within a world model. To foster future research on
  diffusion for world modeling, we release our code, agents and playable world
  models at https://github.com/eloialonso/diamond.
paper_ai_summary: DIAMOND, a reinforcement learning agent using a diffusion world model, achieves top performance on Atari 100k by capturing visual details effectively.
paper_ai_keywords: world models reinforcement learning discrete latent variables diffusion models Atari 100k benchmark mean human normalized score