OmniGlue: Generalizable Feature Matching with Foundation Model Guidance
The image matching field has been witnessing a continuous emergence of novel
learnable feature matching techniques, with ever-improving performance on
conventional benchmarks. However, our investigation shows that despite these
gains, their potential for real-world applications is restricted by their
limited generalization capabilities to novel image domains. In this paper, we
introduce OmniGlue, the first learnable image matcher that is designed with
generalization as a core principle. OmniGlue leverages broad knowledge from a
vision foundation model to guide the feature matching process, boosting
generalization to domains not seen at training time. Additionally, we propose a
novel keypoint position-guided attention mechanism which disentangles spatial
and appearance information, leading to enhanced matching descriptors. We
perform comprehensive experiments on a suite of 7 datasets with varied image
domains, including scene-level, object-centric and aerial images. OmniGlue's
novel components lead to relative gains on unseen domains of 20.9% with
respect to a directly comparable reference model, while also outperforming the
recent LightGlue method by 9.5% relatively.Code and model can be found at
https://hwjiang1510.github.io/OmniGlue
664d777735e245d2ebe86b3c
Hanwen Jiang
claimed_verified
2024-06-13T18:36:07.502Z
6666692aca15c6233f414ac2
/avatars/462841d3ba16d351182e9aa552c36044.svg
Hanwen Jiang
user
hwjiang
664d777735e245d2ebe86b3d
Arjun Karpur
claimed_verified
2024-05-22T20:14:21.489Z
664e2c8300dd56264275bb21
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/UlqJ-gpXxk1pu43TF89DS.jpeg
Arjun Karpur
user
arjunkarpur
664d777735e245d2ebe86b3e
Bingyi Cao
664d777735e245d2ebe86b3f
Qixing Huang
claimed_verified
2025-04-08T07:01:36.814Z
67f18d051358145ee5bb3a11
/avatars/e5a8c12295d7609625c2b891d47c3112.svg
qixing huang
user
qixinghuang1
664d777735e245d2ebe86b40
Andre Araujo
admin_assigned
2024-05-22T09:35:45.156Z
6425ab87ad1e3b0e6e8da2fa
/avatars/0ac9cea8c276ea4fb1c4854c18a86faf.svg
Andre Araujo
user
andrefaraujo
OmniGlue: Generalizable Feature Matching with Foundation Model Guidance
The image matching field has been witnessing a continuous emergence of novel
learnable feature matching techniques, with ever-improving performance on
conventional benchmarks. However, our investigation shows that despite these
gains, their potential for real-world applications is restricted by their
limited generalization capabilities to novel image domains. In this paper, we
introduce OmniGlue, the first learnable image matcher that is designed with
generalization as a core principle. OmniGlue leverages broad knowledge from a
vision foundation model to guide the feature matching process, boosting
generalization to domains not seen at training time. Additionally, we propose a
novel keypoint position-guided attention mechanism which disentangles spatial
and appearance information, leading to enhanced matching descriptors. We
perform comprehensive experiments on a suite of 7 datasets with varied image
domains, including scene-level, object-centric and aerial images. OmniGlue's
novel components lead to relative gains on unseen domains of 20.9% with
respect to a directly comparable reference model, while also outperforming the
recent LightGlue method by 9.5% relatively.Code and model can be found at
https://hwjiang1510.github.io/OmniGlue
OmniGlue, a learnable image matcher, enhances generalization through a vision foundation model and a keypoint position-guided attention mechanism, achieving superior performance on unseen image domains.
learnable feature matching vision foundation model keypoint position-guided attention mechanism generalization feature matching descriptors