title: MINIMA: Modality Invariant Image Matching
summary: Image matching for both cross-view and cross-modality plays a critical role
  in multimodal perception. In practice, the modality gap caused by different
  imaging systems/styles poses great challenges to the matching task. Existing
  works try to extract invariant features for specific modalities and train on
  limited datasets, showing poor generalization. In this paper, we present
  MINIMA, a unified image matching framework for multiple cross-modal cases.
  Without pursuing fancy modules, our MINIMA aims to enhance universal
  performance from the perspective of data scaling up. For such purpose, we
  propose a simple yet effective data engine that can freely produce a large
  dataset containing multiple modalities, rich scenarios, and accurate matching
  labels. Specifically, we scale up the modalities from cheap but rich RGB-only
  matching data, by means of generative models. Under this setting, the matching
  labels and rich diversity of the RGB dataset are well inherited by the
  generated multimodal data. Benefiting from this, we construct MD-syn, a new
  comprehensive dataset that fills the data gap for general multimodal image
  matching. With MD-syn, we can directly train any advanced matching pipeline on
  randomly selected modality pairs to obtain cross-modal ability. Extensive
  experiments on in-domain and zero-shot matching tasks, including 19
  cross-modal cases, demonstrate that our MINIMA can significantly outperform the
  baselines and even surpass modality-specific methods. The dataset and code are
  available at https://github.com/LSXI7/MINIMA .
paper_authors._id: 67893aed91260c834a4d7cd9
paper_authors.name: Xingyu Jiang
paper_authors._id: 67893aed91260c834a4d7cda
paper_authors.name: Jiangwei Ren
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-01-17T17:06:00.673Z
paper_authors._id: 668e05b7410a13fa3dabb2c3
paper_authors.avatarUrl: /avatars/c197ce4957404b2e3223d0ac3031581c.svg
paper_authors.fullname: Jiangwei Ren
paper_authors.type: user
paper_authors.user: lsxi77777
paper_authors._id: 67893aed91260c834a4d7cdb
paper_authors.name: Zizhuo Li
paper_authors._id: 67893aed91260c834a4d7cdc
paper_authors.name: Xin Zhou
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-03-14T08:59:31.315Z
paper_authors._id: 6717c5c36bc2876059ed23ab
paper_authors.avatarUrl: /avatars/52c68fb315760df5ef9323cd8ada5a3c.svg
paper_authors.fullname: Xin Zhou
paper_authors.type: user
paper_authors.user: LMD0311
paper_authors._id: 67893aed91260c834a4d7cdd
paper_authors.name: Dingkang Liang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-16T22:13:25.294Z
paper_authors._id: 67467b5979406f42a14517e9
paper_authors.avatarUrl: /avatars/a8c82dd45b7c94d48ad7fdf8a87bbe66.svg
paper_authors.fullname: Dingkang Liang
paper_authors.type: user
paper_authors.user: dkliang
paper_authors._id: 67893aed91260c834a4d7cde
paper_authors.name: Xiang Bai
paper_title: MINIMA: Modality Invariant Image Matching
paper_summary: Image matching for both cross-view and cross-modality plays a critical role
  in multimodal perception. In practice, the modality gap caused by different
  imaging systems/styles poses great challenges to the matching task. Existing
  works try to extract invariant features for specific modalities and train on
  limited datasets, showing poor generalization. In this paper, we present
  MINIMA, a unified image matching framework for multiple cross-modal cases.
  Without pursuing fancy modules, our MINIMA aims to enhance universal
  performance from the perspective of data scaling up. For such purpose, we
  propose a simple yet effective data engine that can freely produce a large
  dataset containing multiple modalities, rich scenarios, and accurate matching
  labels. Specifically, we scale up the modalities from cheap but rich RGB-only
  matching data, by means of generative models. Under this setting, the matching
  labels and rich diversity of the RGB dataset are well inherited by the
  generated multimodal data. Benefiting from this, we construct MD-syn, a new
  comprehensive dataset that fills the data gap for general multimodal image
  matching. With MD-syn, we can directly train any advanced matching pipeline on
  randomly selected modality pairs to obtain cross-modal ability. Extensive
  experiments on in-domain and zero-shot matching tasks, including 19
  cross-modal cases, demonstrate that our MINIMA can significantly outperform the
  baselines and even surpass modality-specific methods. The dataset and code are
  available at https://github.com/LSXI7/MINIMA .