title: Efficient World Models with Context-Aware Tokenization
summary: Scaling up deep Reinforcement Learning (RL) methods presents a significant
  challenge. Following developments in generative modelling, model-based RL
  positions itself as a strong contender. Recent advances in sequence modelling
  have led to effective transformer-based world models, albeit at the price of
  heavy computations due to the long sequences of tokens required to accurately
  simulate environments. In this work, we propose Delta-IRIS, a new agent with
  a world model architecture composed of a discrete autoencoder that encodes
  stochastic deltas between time steps and an autoregressive transformer that
  predicts future deltas by summarizing the current state of the world with
  continuous tokens. In the Crafter benchmark, Delta-IRIS sets a new state of
  the art at multiple frame budgets, while being an order of magnitude faster to
  train than previous attention-based approaches. We release our code and models
  at https://github.com/vmicheli/delta-iris.
paper_authors._id: 66825dbece294ddc5e7217d5
paper_authors.name: Vincent Micheli
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-01T09:46:10.053Z
paper_authors._id: 607d640fc5e1e416b294ea74
paper_authors.avatarUrl: /avatars/a660039d2152a2ab8139bc3b4a8cb439.svg
paper_authors.fullname: Vincent Micheli
paper_authors.type: user
paper_authors.user: vmicheli
paper_authors._id: 66825dbece294ddc5e7217d6
paper_authors.name: Eloi Alonso
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-01T09:46:16.330Z
paper_authors._id: 66476d39156169f872095c0c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/ja4hmXYNBcprZau6Miux1.png
paper_authors.fullname: Eloi Alonso
paper_authors.type: user
paper_authors.user: eloialonso
paper_authors._id: 66825dbece294ddc5e7217d7
paper_authors.name: François Fleuret
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-07-01T09:46:22.087Z
paper_authors._id: 6318c37e69ef5467e18fc63f
paper_authors.avatarUrl: /avatars/7f1ac593608a89926f4107b4c3dd91d4.svg
paper_authors.fullname: François Fleuret
paper_authors.type: user
paper_authors.user: francoisfleuret
paper_title: Efficient World Models with Context-Aware Tokenization
paper_summary: Scaling up deep Reinforcement Learning (RL) methods presents a significant
  challenge. Following developments in generative modelling, model-based RL
  positions itself as a strong contender. Recent advances in sequence modelling
  have led to effective transformer-based world models, albeit at the price of
  heavy computations due to the long sequences of tokens required to accurately
  simulate environments. In this work, we propose Delta-IRIS, a new agent with
  a world model architecture composed of a discrete autoencoder that encodes
  stochastic deltas between time steps and an autoregressive transformer that
  predicts future deltas by summarizing the current state of the world with
  continuous tokens. In the Crafter benchmark, Delta-IRIS sets a new state of
  the art at multiple frame budgets, while being an order of magnitude faster to
  train than previous attention-based approaches. We release our code and models
  at https://github.com/vmicheli/delta-iris.
paper_ai_summary: A new agent, $\Delta$-IRIS, achieves state-of-the-art performance in model-based reinforcement learning by using a discrete autoencoder and autoregressive transformer architecture, resulting in faster training and improved simulation efficiency.
paper_ai_keywords: Reinforcement Learning (RL) model-based RL sequence modelling transformer-based world models discrete autoencoder autoregressive transformer Crafter benchmark