4D-LRM: Large Space-Time Reconstruction Model From and To Any View at
  Any Time
Can we scale 4D pretraining to learn general space-time representations that
reconstruct an object from a few views at some times to any view at any time?
We provide an affirmative answer with 4D-LRM, the first large-scale 4D
reconstruction model that takes input from unconstrained views and timestamps
and renders arbitrary novel view-time combinations. Unlike prior 4D approaches,
e.g., optimization-based, geometry-based, or generative, that struggle with
efficiency, generalization, or faithfulness, 4D-LRM learns a unified space-time
representation and directly predicts per-pixel 4D Gaussian primitives from
posed image tokens across time, enabling fast, high-quality rendering at, in
principle, infinite frame rate. Our results demonstrate that scaling
spatiotemporal pretraining enables accurate and efficient 4D reconstruction. We
show that 4D-LRM generalizes to novel objects, interpolates across time, and
handles diverse camera setups. It reconstructs 24-frame sequences in one
forward pass with less than 1.5 seconds on a single A100 GPU.
685b168dd2ee4fac76521d68
Ziqiao Ma
claimed_verified
2025-06-25T08:09:00.720Z
630cfc45b66f088d547b2768
https://cdn-avatars.huggingface.co/v1/production/uploads/630cfc45b66f088d547b2768/ge2BZLDYeTlpnlIlraHNw.png
Martin Ziqiao Ma
user
marstin
685b168dd2ee4fac76521d69
Xuweiyi Chen
685b168dd2ee4fac76521d6a
Shoubin Yu
685b168dd2ee4fac76521d6b
Sai Bi
685b168dd2ee4fac76521d6c
Kai Zhang
685b168dd2ee4fac76521d6d
Chen Ziwen
685b168dd2ee4fac76521d6e
Sihan Xu
685b168dd2ee4fac76521d6f
Jianing Yang
685b168dd2ee4fac76521d70
Zexiang Xu
685b168dd2ee4fac76521d71
Kalyan Sunkavalli
685b168dd2ee4fac76521d72
Mohit Bansal
685b168dd2ee4fac76521d73
Joyce Chai
685b168dd2ee4fac76521d74
Hao Tan
4D-LRM: Large Space-Time Reconstruction Model From and To Any View at
  Any Time
Can we scale 4D pretraining to learn general space-time representations that
reconstruct an object from a few views at some times to any view at any time?
We provide an affirmative answer with 4D-LRM, the first large-scale 4D
reconstruction model that takes input from unconstrained views and timestamps
and renders arbitrary novel view-time combinations. Unlike prior 4D approaches,
e.g., optimization-based, geometry-based, or generative, that struggle with
efficiency, generalization, or faithfulness, 4D-LRM learns a unified space-time
representation and directly predicts per-pixel 4D Gaussian primitives from
posed image tokens across time, enabling fast, high-quality rendering at, in
principle, infinite frame rate. Our results demonstrate that scaling
spatiotemporal pretraining enables accurate and efficient 4D reconstruction. We
show that 4D-LRM generalizes to novel objects, interpolates across time, and
handles diverse camera setups. It reconstructs 24-frame sequences in one
forward pass with less than 1.5 seconds on a single A100 GPU.
4D-LRM is a large-scale model that efficiently reconstructs objects from multiple views and times into any view-time combination using space-time representations and Gaussian primitives.
4D-LRM space-time representation Gaussian primitives posed image tokens 4D reconstruction frame rate