SPREADING VECTORS FOR SIMILARITY SEARCH
Discretizing multi-dimensional data distributions is a fundamental step of modern indexing methods. State-of-the-art techniques learn parameters of quantizers on training data for optimal performance, thus adapting quantizers to the data. In this work, we propose to reverse this paradigm and adapt the data to the quantizer: we train a neural net which last layer forms a fixed parameter-free quantizer, such as pre-defined points of a hyper-sphere. As a proxy objective, we design and train a neural network that favors uniformity in the spherical latent space, while preserving the neighborhood structure after the mapping. We propose a new regularizer derived from the Kozachenko-Leonenko differential entropy estimator to enforce uniformity and combine it with a locality-aware triplet loss. Experiments show that our end-to-end approach outperforms most learned quantization methods, and is competitive with the state of the art on widely adopted benchmarks. Furthermore, we show that training without the quantization step results in almost no difference in accuracy, but yields a generic catalyzer that can be applied with any subsequent quantizer. The code is available online 1 .
Pros
----
[Originality]
The authors propose a novel idea of learning representations that improves the performance of the subsequent fixed discretization method.
[Clarity]
The authors clearly motivate their solution and explain the different ideas and enhancements introduced. The manuscript is fairly easy to follow. The different terms in the optimization problem are clearly explained and their individual behaviour are presented for the better understanding.
[Significance]
The empirical results for the proposed scheme are compared against various baselines under various scenarios and the results demonstrate the significant utility of the proposed scheme.
Limitations
-----------
[Clarity]
The training times for the catalyzer is never discussed in this manuscript (even relative to the training times of the considered baselines). Moreover, it is not clear if the inference time of the catalyzer is included in the results such as Table 1. Even if, PQ and the catalyzer+lattice might have comparable search recalls, it would be good to understand the relative search times to get similar accuracy especially since the inference time for the catalyzer (which is part of the search time) can be fairly significant.
[Clarity/Significance]
One important point not discussed in this manuscript is the choice of the structure (architechture) of the catalyzer. Is the catalyzer architecture dependent on the data?
  - If yes, how to find an appropriate architecture?
  - If no, what is it about the proposed architecture that makes it sufficient for all data sets?
In my opinion, this is extremely important since this drives the applicability of the proposed scheme beyond the presented examples.
[Minor question]
- Is the parameter r in the rank loss same as the norm r in the lattice quantizer? This is a bit confusing.