title: Video Occupancy Models
summary: We introduce a new family of video prediction models designed to support
  downstream control tasks. We call these models Video Occupancy models (VOCs).
  VOCs operate in a compact latent space, thus avoiding the need to make
  predictions about individual pixels. Unlike prior latent-space world models,
  VOCs directly predict the discounted distribution of future states in a single
  step, thus avoiding the need for multistep roll-outs. We show that both
  properties are beneficial when building predictive models of video for use in
  downstream control. Code is available at
  https://github.com/manantomar/video-occupancy-models{github.com/manantomar/video-occupancy-models}.
paper_authors._id: 6695e1627c6c4ffc86b928a3
paper_authors.name: Manan Tomar
paper_authors._id: 6695e1627c6c4ffc86b928a4
paper_authors.name: Philippe Hansen-Estruch
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-12-17T08:05:02.525Z
paper_authors._id: 644eb923a8f384abd038a4f7
paper_authors.avatarUrl: /avatars/3d1a29c6149a6ffc482842f1438ba55a.svg
paper_authors.fullname: Philippe I Hansen-Estruch
paper_authors.type: user
paper_authors.user: philippehansen
paper_authors._id: 6695e1627c6c4ffc86b928a5
paper_authors.name: Philip Bachman
paper_authors._id: 6695e1627c6c4ffc86b928a6
paper_authors.name: Alex Lamb
paper_authors._id: 6695e1627c6c4ffc86b928a7
paper_authors.name: John Langford
paper_authors._id: 6695e1627c6c4ffc86b928a8
paper_authors.name: Matthew E. Taylor
paper_authors._id: 6695e1627c6c4ffc86b928a9
paper_authors.name: Sergey Levine
paper_title: Video Occupancy Models
paper_summary: We introduce a new family of video prediction models designed to support
  downstream control tasks. We call these models Video Occupancy models (VOCs).
  VOCs operate in a compact latent space, thus avoiding the need to make
  predictions about individual pixels. Unlike prior latent-space world models,
  VOCs directly predict the discounted distribution of future states in a single
  step, thus avoiding the need for multistep roll-outs. We show that both
  properties are beneficial when building predictive models of video for use in
  downstream control. Code is available at
  https://github.com/manantomar/video-occupancy-models{github.com/manantomar/video-occupancy-models}.
paper_ai_summary: Video Occupancy models predict future video states in a compact latent space with a single-step, discounted distribution, benefiting downstream control tasks.
paper_ai_keywords: Video Occupancy models VOCs latent space discounted distribution single-step predictions world models multistep roll-outs predictive models downstream control