title: OmniGlue: Generalizable Feature Matching with Foundation Model Guidance
summary: The image matching field has been witnessing a continuous emergence of novel
  learnable feature matching techniques, with ever-improving performance on
  conventional benchmarks. However, our investigation shows that despite these
  gains, their potential for real-world applications is restricted by their
  limited generalization capabilities to novel image domains. In this paper, we
  introduce OmniGlue, the first learnable image matcher that is designed with
  generalization as a core principle. OmniGlue leverages broad knowledge from a
  vision foundation model to guide the feature matching process, boosting
  generalization to domains not seen at training time. Additionally, we propose a
  novel keypoint position-guided attention mechanism which disentangles spatial
  and appearance information, leading to enhanced matching descriptors. We
  perform comprehensive experiments on a suite of 7 datasets with varied image
  domains, including scene-level, object-centric and aerial images. OmniGlue's
  novel components lead to relative gains on unseen domains of 20.9% with
  respect to a directly comparable reference model, while also outperforming the
  recent LightGlue method by 9.5% relatively.Code and model can be found at
  https://hwjiang1510.github.io/OmniGlue
paper_authors._id: 664d777735e245d2ebe86b3c
paper_authors.name: Hanwen Jiang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-06-13T18:36:07.502Z
paper_authors._id: 6666692aca15c6233f414ac2
paper_authors.avatarUrl: /avatars/462841d3ba16d351182e9aa552c36044.svg
paper_authors.fullname: Hanwen Jiang
paper_authors.type: user
paper_authors.user: hwjiang
paper_authors._id: 664d777735e245d2ebe86b3d
paper_authors.name: Arjun Karpur
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-05-22T20:14:21.489Z
paper_authors._id: 664e2c8300dd56264275bb21
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/UlqJ-gpXxk1pu43TF89DS.jpeg
paper_authors.fullname: Arjun Karpur
paper_authors.type: user
paper_authors.user: arjunkarpur
paper_authors._id: 664d777735e245d2ebe86b3e
paper_authors.name: Bingyi Cao
paper_authors._id: 664d777735e245d2ebe86b3f
paper_authors.name: Qixing Huang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-08T07:01:36.814Z
paper_authors._id: 67f18d051358145ee5bb3a11
paper_authors.avatarUrl: /avatars/e5a8c12295d7609625c2b891d47c3112.svg
paper_authors.fullname: qixing huang
paper_authors.type: user
paper_authors.user: qixinghuang1
paper_authors._id: 664d777735e245d2ebe86b40
paper_authors.name: Andre Araujo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-05-22T09:35:45.156Z
paper_authors._id: 6425ab87ad1e3b0e6e8da2fa
paper_authors.avatarUrl: /avatars/0ac9cea8c276ea4fb1c4854c18a86faf.svg
paper_authors.fullname: Andre Araujo
paper_authors.type: user
paper_authors.user: andrefaraujo
paper_title: OmniGlue: Generalizable Feature Matching with Foundation Model Guidance
paper_summary: The image matching field has been witnessing a continuous emergence of novel
  learnable feature matching techniques, with ever-improving performance on
  conventional benchmarks. However, our investigation shows that despite these
  gains, their potential for real-world applications is restricted by their
  limited generalization capabilities to novel image domains. In this paper, we
  introduce OmniGlue, the first learnable image matcher that is designed with
  generalization as a core principle. OmniGlue leverages broad knowledge from a
  vision foundation model to guide the feature matching process, boosting
  generalization to domains not seen at training time. Additionally, we propose a
  novel keypoint position-guided attention mechanism which disentangles spatial
  and appearance information, leading to enhanced matching descriptors. We
  perform comprehensive experiments on a suite of 7 datasets with varied image
  domains, including scene-level, object-centric and aerial images. OmniGlue's
  novel components lead to relative gains on unseen domains of 20.9% with
  respect to a directly comparable reference model, while also outperforming the
  recent LightGlue method by 9.5% relatively.Code and model can be found at
  https://hwjiang1510.github.io/OmniGlue
paper_ai_summary: OmniGlue, a learnable image matcher, enhances generalization through a vision foundation model and a keypoint position-guided attention mechanism, achieving superior performance on unseen image domains.
paper_ai_keywords: learnable feature matching vision foundation model keypoint position-guided attention mechanism generalization feature matching descriptors