Local All-Pair Correspondence for Point Tracking
We introduce LocoTrack, a highly accurate and efficient model designed for
the task of tracking any point (TAP) across video sequences. Previous
approaches in this task often rely on local 2D correlation maps to establish
correspondences from a point in the query image to a local region in the target
image, which often struggle with homogeneous regions or repetitive features,
leading to matching ambiguities. LocoTrack overcomes this challenge with a
novel approach that utilizes all-pair correspondences across regions, i.e.,
local 4D correlation, to establish precise correspondences, with bidirectional
correspondence and matching smoothness significantly enhancing robustness
against ambiguities. We also incorporate a lightweight correlation encoder to
enhance computational efficiency, and a compact Transformer architecture to
integrate long-term temporal information. LocoTrack achieves unmatched accuracy
on all TAP-Vid benchmarks and operates at a speed almost 6 times faster than
the current state-of-the-art.
669f3f5a3e173b3293dfafc0
Seokju Cho
claimed_verified
2024-07-24T07:27:15.383Z
602e45160daeb0df2a81b244
/avatars/f6bf69f0c1342f8cfad05d5775e59bf4.svg
Seokju Cho
user
hamacojr
669f3f5a3e173b3293dfafc1
Jiahui Huang
admin_assigned
2024-07-23T12:55:40.749Z
644a717e75fce8ebef4e4955
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/zLga4NZBohFPlv50dcAo9.png
Jiahui Huang
user
heiwang1997
669f3f5a3e173b3293dfafc2
Jisu Nam
admin_assigned
2024-07-23T12:55:55.864Z
637c7420f219c71f93ec8f81
/avatars/969b72bd4320423af89e6a5d0ffa03cc.svg
frog
user
frog123123123123
669f3f5a3e173b3293dfafc3
Honggyu An
669f3f5a3e173b3293dfafc4
Seungryong Kim
admin_assigned
2024-07-23T12:56:10.746Z
65cf717450818a335a1d3021
/avatars/382a0e0f40f661cda1b2531e3e6ea2ee.svg
Seungryong Kim
user
seungryong
669f3f5a3e173b3293dfafc5
Joon-Young Lee
admin_assigned
2024-07-23T12:56:17.205Z
64b76c7990b38df83381824b
/avatars/0a5bec2ea480fb3f43c4b24d59c50e81.svg
Joon-Young Lee
user
joonyounglee
Local All-Pair Correspondence for Point Tracking
We introduce LocoTrack, a highly accurate and efficient model designed for
the task of tracking any point (TAP) across video sequences. Previous
approaches in this task often rely on local 2D correlation maps to establish
correspondences from a point in the query image to a local region in the target
image, which often struggle with homogeneous regions or repetitive features,
leading to matching ambiguities. LocoTrack overcomes this challenge with a
novel approach that utilizes all-pair correspondences across regions, i.e.,
local 4D correlation, to establish precise correspondences, with bidirectional
correspondence and matching smoothness significantly enhancing robustness
against ambiguities. We also incorporate a lightweight correlation encoder to
enhance computational efficiency, and a compact Transformer architecture to
integrate long-term temporal information. LocoTrack achieves unmatched accuracy
on all TAP-Vid benchmarks and operates at a speed almost 6 times faster than
the current state-of-the-art.
LocoTrack enhances video point tracking using 4D correlation and a compact Transformer to achieve high accuracy and speed.
local 4D correlation bidirectional correspondence correlation encoder Transformer architecture TAP-Vid benchmarks