Video Occupancy Models
We introduce a new family of video prediction models designed to support
downstream control tasks. We call these models Video Occupancy models (VOCs).
VOCs operate in a compact latent space, thus avoiding the need to make
predictions about individual pixels. Unlike prior latent-space world models,
VOCs directly predict the discounted distribution of future states in a single
step, thus avoiding the need for multistep roll-outs. We show that both
properties are beneficial when building predictive models of video for use in
downstream control. Code is available at
https://github.com/manantomar/video-occupancy-models{github.com/manantomar/video-occupancy-models}.
6695e1627c6c4ffc86b928a3
Manan Tomar
6695e1627c6c4ffc86b928a4
Philippe Hansen-Estruch
claimed_verified
2024-12-17T08:05:02.525Z
644eb923a8f384abd038a4f7
/avatars/3d1a29c6149a6ffc482842f1438ba55a.svg
Philippe I Hansen-Estruch
user
philippehansen
6695e1627c6c4ffc86b928a5
Philip Bachman
6695e1627c6c4ffc86b928a6
Alex Lamb
6695e1627c6c4ffc86b928a7
John Langford
6695e1627c6c4ffc86b928a8
Matthew E. Taylor
6695e1627c6c4ffc86b928a9
Sergey Levine
Video Occupancy Models
We introduce a new family of video prediction models designed to support
downstream control tasks. We call these models Video Occupancy models (VOCs).
VOCs operate in a compact latent space, thus avoiding the need to make
predictions about individual pixels. Unlike prior latent-space world models,
VOCs directly predict the discounted distribution of future states in a single
step, thus avoiding the need for multistep roll-outs. We show that both
properties are beneficial when building predictive models of video for use in
downstream control. Code is available at
https://github.com/manantomar/video-occupancy-models{github.com/manantomar/video-occupancy-models}.
Video Occupancy models predict future video states in a compact latent space with a single-step, discounted distribution, benefiting downstream control tasks.
Video Occupancy models VOCs latent space discounted distribution single-step predictions world models multistep roll-outs predictive models downstream control