CorrMAE: Pre-training Correspondence Transformers with Masked Autoencoder
<|reference_start|>CorrMAE: Pre-training Correspondence Transformers with Masked Autoencoder: Pre-training has emerged as a simple yet powerful methodology for representation learning across various domains. However, due to the expensive training cost and limited data, pre-training has not yet been extensively studied in correspondence pruning. To tackle these challenges, we propose a pre-training method to acquire a generic inliers-consistent representation by reconstructing masked correspondences, providing a strong initial representation for downstream tasks. Toward this objective, a modicum of true correspondences naturally serve as input, thus significantly reducing pre-training overhead. In practice, we introduce CorrMAE, an extension of the mask autoencoder framework tailored for the pre-training of correspondence pruning. CorrMAE involves two main phases, \ie correspondence learning and matching point reconstruction, guiding the reconstruction of masked correspondences through learning visible correspondence consistency. Herein, we employ a dual-branch structure with an ingenious positional encoding to reconstruct unordered and irregular correspondences. Also, a bi-level designed encoder is proposed for correspondence learning, which offers enhanced consistency learning capability and transferability. Extensive experiments have shown that the model pre-trained with our CorrMAE outperforms prior work on multiple challenging benchmarks. Meanwhile, our CorrMAE is primarily a task-driven pre-training method, and can achieve notable improvements for downstream tasks by pre-training on the targeted dataset. We hope this work can provide a starting point for correspondence pruning pre-training.<|reference_end|>
@article{liao2024corrmae:,
    title={CorrMAE: Pre-training Correspondence Transformers with Masked
  Autoencoder},
    author={Tangfei Liao, Xiaoqin Zhang, Guobao Xiao, Min Li, Tao Wang, and Mang
  Ye},
    journal={arXiv preprint arXiv:2406.05773},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.05773},
    primaryClass={cs.CV}
}