BinGAN: Learning Compact Binary Descriptors with a Regularized GAN
In this paper, we propose a novel regularization method for Generative Adversarial Networks, which allows the model to learn discriminative yet compact binary representations of image patches (image descriptors). We employ the dimensionality reduction that takes place in the intermediate layers of the discriminator network and train binarized low-dimensional representation of the penultimate layer to mimic the distribution of the higher-dimensional preceding layers. To achieve this, we introduce two loss terms that aim at: (i) reducing the correlation between the dimensions of the binarized low-dimensional representation of the penultimate layer (i.e. maximizing joint entropy) and (ii) propagating the relations between the dimensions in the high-dimensional space to the low-dimensional space. We evaluate the resulting binary image descriptors on two challenging applications, image matching and retrieval, and achieve state-of-the-art results.
Summary This paper proposes a variant of GAN to learn compact binary descriptors for image patch matching. The authors introduce two novel regularizers to propagate Hamming distance between two layers in the discriminator and encourage the diversity of learned descriptors. The presentation is easy to follow, and the method is validated by benchmark datasets.   Major concerns: [Motivation and Presentation] First of all, it’s not so clear the reason why adversarial training helps to learn compact binary descriptors. In addition, the motivation on DMR is also not fully addressed in my sense. In my understanding, the discriminator has two binary representation layers; one of them has the larger number of bits and the other is used for the compact binary descriptor. However, it’s not clearly described in the current draft.  [About Experiments] There exists a simple baseline by just combining state-of-the-art image descriptors with binary embedding algorithms. Specifically, in stead of learning descriptors and its binary representation together, let us split two parts and use SOTA algorithms in both directions. For example, what about using the following algorithm followed by deep hashing to generate compact binary descriptors? - Image Patch Matching Using Convolutional Descriptors with Euclidean Distance, ACCV’16 workshop, https://arxiv.org/abs/1710.11359 I believe that there exist more follow-up works to learn more meaningful descriptors for matching image patches.  Preliminary rating: borderline This paper has some potential to be a good application of GAN, which learns compact binary descriptors for image matching. Yet, the motivation on network architecture and experiments are not sufficient. Please elaborate them in the rebuttal, if possible.  == Updated review == The rebuttal addressed my concerns on the experiment results. Yet, I still think that the technical novelty on the proposed methods is not strong for NIPS. Specifically, the entropy regularizer has been studied to learn decorrelated hash bits before;  - SPEC Hashing: Similarity Preserving algorithm for Entropy-based Coding, CVPR’10. Though the exact form used in this manuscript might not been employed before, it is quite common to learn decorrelated bits by maximizing entropy on them.   Moreover, the other regularizer (DMR) is designed to reduce the dimensionality of high-dimensional binary codes learned from an intermediate layer of discriminator. It’s a very natural and practical regularizer, but I’m not sure that it’s solely enough to be a nips paper. Therefore, I haven’t changed my original score (5).