title: MatchAnything: Universal Cross-Modality Image Matching with Large-Scale
    Pre-Training
summary: Image matching, which aims to identify corresponding pixel locations between
  images, is crucial in a wide range of scientific disciplines, aiding in image
  registration, fusion, and analysis. In recent years, deep learning-based image
  matching algorithms have dramatically outperformed humans in rapidly and
  accurately finding large amounts of correspondences. However, when dealing with
  images captured under different imaging modalities that result in significant
  appearance changes, the performance of these algorithms often deteriorates due
  to the scarcity of annotated cross-modal training data. This limitation hinders
  applications in various fields that rely on multiple image modalities to obtain
  complementary information. To address this challenge, we propose a large-scale
  pre-training framework that utilizes synthetic cross-modal training signals,
  incorporating diverse data from various sources, to train models to recognize
  and match fundamental structures across images. This capability is transferable
  to real-world, unseen cross-modality image matching tasks. Our key finding is
  that the matching model trained with our framework achieves remarkable
  generalizability across more than eight unseen cross-modality registration
  tasks using the same network weight, substantially outperforming existing
  methods, whether designed for generalization or tailored for specific tasks.
  This advancement significantly enhances the applicability of image matching
  technologies across various scientific disciplines and paves the way for new
  applications in multi-modality human and artificial intelligence analysis and
  beyond.
paper_authors._id: 67880298f2b3aef29aea2777
paper_authors.name: Xingyi He
paper_authors._id: 67880298f2b3aef29aea2778
paper_authors.name: Hao Yu
paper_authors._id: 67880298f2b3aef29aea2779
paper_authors.name: Sida Peng
paper_authors._id: 67880298f2b3aef29aea277a
paper_authors.name: Dongli Tan
paper_authors._id: 67880298f2b3aef29aea277b
paper_authors.name: Zehong Shen
paper_authors._id: 67880298f2b3aef29aea277c
paper_authors.name: Hujun Bao
paper_authors._id: 67880298f2b3aef29aea277d
paper_authors.name: Xiaowei Zhou
paper_title: MatchAnything: Universal Cross-Modality Image Matching with Large-Scale
    Pre-Training
paper_summary: Image matching, which aims to identify corresponding pixel locations between
  images, is crucial in a wide range of scientific disciplines, aiding in image
  registration, fusion, and analysis. In recent years, deep learning-based image
  matching algorithms have dramatically outperformed humans in rapidly and
  accurately finding large amounts of correspondences. However, when dealing with
  images captured under different imaging modalities that result in significant
  appearance changes, the performance of these algorithms often deteriorates due
  to the scarcity of annotated cross-modal training data. This limitation hinders
  applications in various fields that rely on multiple image modalities to obtain
  complementary information. To address this challenge, we propose a large-scale
  pre-training framework that utilizes synthetic cross-modal training signals,
  incorporating diverse data from various sources, to train models to recognize
  and match fundamental structures across images. This capability is transferable
  to real-world, unseen cross-modality image matching tasks. Our key finding is
  that the matching model trained with our framework achieves remarkable
  generalizability across more than eight unseen cross-modality registration
  tasks using the same network weight, substantially outperforming existing
  methods, whether designed for generalization or tailored for specific tasks.
  This advancement significantly enhances the applicability of image matching
  technologies across various scientific disciplines and paves the way for new
  applications in multi-modality human and artificial intelligence analysis and
  beyond.
paper_ai_summary: A pre-training framework using synthetic cross-modal signals enhances the generalizability of deep learning image matching across different imaging modalities.
paper_ai_keywords: image matching deep learning cross-modal training pre-training synthetic cross-modal training signals fundamental structures generalizability cross-modality registration tasks