Doppelgangers: Learning to Disambiguate Images of Similar Structures
We consider the visual disambiguation task of determining whether a pair of
visually similar images depict the same or distinct 3D surfaces (e.g., the same
or opposite sides of a symmetric building). Illusory image matches, where two
images observe distinct but visually similar 3D surfaces, can be challenging
for humans to differentiate, and can also lead 3D reconstruction algorithms to
produce erroneous results. We propose a learning-based approach to visual
disambiguation, formulating it as a binary classification task on image pairs.
To that end, we introduce a new dataset for this problem, Doppelgangers, which
includes image pairs of similar structures with ground truth labels. We also
design a network architecture that takes the spatial distribution of local
keypoints and matches as input, allowing for better reasoning about both local
and global cues. Our evaluation shows that our method can distinguish illusory
matches in difficult cases, and can be integrated into SfM pipelines to produce
correct, disambiguated 3D reconstructions. See our project page for our code,
datasets, and more results: http://doppelgangers-3d.github.io/.
64f836239433a36edf4f482c
Ruojin Cai
64f836239433a36edf4f482d
Joseph Tung
claimed_verified
2024-01-10T08:41:21.483Z
656d6c26996819a828cbb55f
/avatars/cbc80b438201752cc1ca45edc422bc50.svg
Joseph Tung
user
jotjot
64f836239433a36edf4f482e
Qianqian Wang
64f836239433a36edf4f482f
Hadar Averbuch-Elor
64f836239433a36edf4f4830
Bharath Hariharan
64f836239433a36edf4f4831
Noah Snavely
Doppelgangers: Learning to Disambiguate Images of Similar Structures
We consider the visual disambiguation task of determining whether a pair of
visually similar images depict the same or distinct 3D surfaces (e.g., the same
or opposite sides of a symmetric building). Illusory image matches, where two
images observe distinct but visually similar 3D surfaces, can be challenging
for humans to differentiate, and can also lead 3D reconstruction algorithms to
produce erroneous results. We propose a learning-based approach to visual
disambiguation, formulating it as a binary classification task on image pairs.
To that end, we introduce a new dataset for this problem, Doppelgangers, which
includes image pairs of similar structures with ground truth labels. We also
design a network architecture that takes the spatial distribution of local
keypoints and matches as input, allowing for better reasoning about both local
and global cues. Our evaluation shows that our method can distinguish illusory
matches in difficult cases, and can be integrated into SfM pipelines to produce
correct, disambiguated 3D reconstructions. See our project page for our code,
datasets, and more results: http://doppelgangers-3d.github.io/.
A learning-based approach using a custom dataset and network architecture distinguishes illusory image matches, improving 3D reconstruction accuracy.
binary classification image pairs Doppelgangers dataset spatial distribution local keypoints global cues SfM pipelines