Neural Eulerian Scene Flow Fields
Kyle Vedder Neehar Peri Ishan Khatri Siyi Li Eric Eaton Mehmet Kemal Kocamaz Yue Wang Zhiding Yu Deva Ramanan Joachim Pehserl
We reframe scene flow as the task of estimating a continuous space-time ordinary differential equation (ODE) that describes motion for an entire observation sequence, represented with a neural prior. Our method, EulerFlow, optimizes this neural prior estimate against several multi-observation reconstruction objectives, enabling high quality scene flow estimation via self-supervision on real-world data. EulerFlow works out-of-the-box without tuning across multiple domains, including large-scale autonomous driving scenes and dynamic tabletop settings. Remarkably, EulerFlow produces high quality flow estimates on small, fast moving objects like birds and tennis balls, and exhibits emergent 3D point tracking behavior by solving its estimated ODE over long-time horizons. On the Argoverse 2 2024 Scene Flow Challenge, EulerFlow outperforms all prior art, surpassing the next-best unsupervised method by more than 2.5 times, and even exceeding the next-best supervised method by over 10%. See https://vedder.io/eulerflow for interactive visuals.
Scene Flow, Neural Prior, Ordinary Differential Equation, Reconstruction
We model scene flow as an estimating a ODE over many observations; our unsupervised method is high quality (SotA on important benchmarks) and works out-of-the-box on many diverse domains.