Tracking Everything Everywhere All at Once
We present a new test-time optimization method for estimating dense and
long-range motion from a video sequence. Prior optical flow or particle video
tracking algorithms typically operate within limited temporal windows,
struggling to track through occlusions and maintain global consistency of
estimated motion trajectories. We propose a complete and globally consistent
motion representation, dubbed OmniMotion, that allows for accurate, full-length
motion estimation of every pixel in a video. OmniMotion represents a video
using a quasi-3D canonical volume and performs pixel-wise tracking via
bijections between local and canonical space. This representation allows us to
ensure global consistency, track through occlusions, and model any combination
of camera and object motion. Extensive evaluations on the TAP-Vid benchmark and
real-world footage show that our approach outperforms prior state-of-the-art
methods by a large margin both quantitatively and qualitatively. See our
project page for more results: http://omnimotion.github.io/
6482c29c99110097e5490fe1
Qianqian Wang
6482c29c99110097e5490fe2
Yen-Yu Chang
admin_assigned
2023-06-12T13:04:56.683Z
638b94ffdaa4cd1d6b40d719
/avatars/1cb7f8d4062838710365b79b2f1d595a.svg
Yen-Yu Chang
user
KAXY
6482c29c99110097e5490fe3
Ruojin Cai
6482c29c99110097e5490fe4
Zhengqi Li
6482c29c99110097e5490fe5
Bharath Hariharan
6482c29c99110097e5490fe6
Aleksander Holynski
admin_assigned
2023-06-12T13:05:46.990Z
62f6e2792e53c2efd33faa92
/avatars/85d94b4022577747b8d2d10a82c2f3c7.svg
Aleksander Holynski
user
holynski
6482c29c99110097e5490fe7
Noah Snavely
admin_assigned
2023-06-12T13:06:03.310Z
632b79ca0a9dc02985af9e0b
/avatars/e2ca585398a3212b2956a14804d0ba67.svg
Noah Snavely
user
jimantha
Tracking Everything Everywhere All at Once
We present a new test-time optimization method for estimating dense and
long-range motion from a video sequence. Prior optical flow or particle video
tracking algorithms typically operate within limited temporal windows,
struggling to track through occlusions and maintain global consistency of
estimated motion trajectories. We propose a complete and globally consistent
motion representation, dubbed OmniMotion, that allows for accurate, full-length
motion estimation of every pixel in a video. OmniMotion represents a video
using a quasi-3D canonical volume and performs pixel-wise tracking via
bijections between local and canonical space. This representation allows us to
ensure global consistency, track through occlusions, and model any combination
of camera and object motion. Extensive evaluations on the TAP-Vid benchmark and
real-world footage show that our approach outperforms prior state-of-the-art
methods by a large margin both quantitatively and qualitatively. See our
project page for more results: http://omnimotion.github.io/
OmniMotion, a globally consistent motion representation using a quasi-3D canonical volume, outperforms state-of-the-art methods in motion estimation by modeling camera and object motion through occlusions.
optical flow particle video tracking global consistency motion representation quasi-3D canonical volume pixel-wise tracking bijections TAP-Vid benchmark