Permutation Diffusion Maps (PDM) with Application to the Image Association Problem in Computer Vision
Consistently matching keypoints across images, and the related problem of finding clusters of nearby images, are critical components of various tasks in Computer Vision, including Structure from Motion (SfM). Unfortunately, occlusion and large repetitive structures tend to mislead most currently used matching algorithms, leading to characteristic pathologies in the final output. In this paper we introduce a new method, Permutations Diffusion Maps (PDM), to solve the matching problem, as well as a related new affinity measure, derived using ideas from harmonic analysis on the symmetric group. We show that just by using it as a preprocessing step to existing SfM pipelines, PDM can greatly improve reconstruction quality on difficult datasets.
This paper applies the diffusion map framework in a permutation setting similar to the cited work [23] which works with rotations in primarily image association problems. The goal is to develop a global association scheme that can better handle the presence of duplicate structures. The authors support their proposed method with experiments on various real image datasets.This is a clearly written paper and the reviewer expect it to be interesting to the NIPS audience. This reviewer sees the theoretical contribution as the idea of viewing the transformations that need to be coherent as a group and the observation that a preprocessing can bring benefit to existing intricate algorithms and implementations.However there is a concern also in this regard. One may argue that the idea of thinking in terms of an algebraic group is essentially present already in the cited [23]. The authors describe the case when the transformations are matching and can be viewed as permutations, they derive the formulas but also point out that they are very similar to those of in the rotation setting.As a result this reviewer is somewhat undecided, but the strong experimental section tipped the balance in favor of recommending acceptance.A few detailed comments:The authors should consider citing: Coifman, R.R.; S. Lafon. (2006). "Diffusion maps". Applied and Computational Harmonic Analysis, that introduced diffusion maps.Line 135: typo in x_p^i \sim x_q^jLine 150: it may make sense that at the first use of "group" the authors would use "algebraic group" considering the audience and venue (not a group theory conference)Line 209-210: The Hermitian conjugate is traditionally denoted by M^*. This reviewer sees no reason to deviate from this custom. Also note the transpose is typically marked by capital T, incorrect in (1).Line 211: Define compact group. Why we need compact?