DINO-Tracker: Taming DINO for Self-Supervised Point Tracking in a Single Video
<|reference_start|>DINO-Tracker: Taming DINO for Self-Supervised Point Tracking in a Single Video: We present DINO-Tracker -- a new framework for long-term dense tracking in video. The pillar of our approach is combining test-time training on a single video, with the powerful localized semantic features learned by a pre-trained DINO-ViT model. Specifically, our framework simultaneously adopts DINO's features to fit to the motion observations of the test video, while training a tracker that directly leverages the refined features. The entire framework is trained end-to-end using a combination of self-supervised losses, and regularization that allows us to retain and benefit from DINO's semantic prior. Extensive evaluation demonstrates that our method achieves state-of-the-art results on known benchmarks. DINO-tracker significantly outperforms self-supervised methods and is competitive with state-of-the-art supervised trackers, while outperforming them in challenging cases of tracking under long-term occlusions.<|reference_end|>
@article{tumanyan2024dino-tracker:,
    title={DINO-Tracker: Taming DINO for Self-Supervised Point Tracking in a Single
  Video},
    author={Narek Tumanyan, Assaf Singer, Shai Bagon, Tali Dekel},
    journal={arXiv preprint arXiv:2403.14548},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.14548},
    primaryClass={cs.CV}
}