Learning invariant representations and applications to face verification
One approach to computer object recognition and modeling the brain's ventral stream involves unsupervised learning of representations that are invariant to common transformations. However, applications of these ideas have usually been limited to 2D affine transformations, e.g., translation and scaling, since they are easiest to solve via convolution. In accord with a recent theory of transformationinvariance [1], we propose a model that, while capturing other common convolutional networks as special cases, can also be used with arbitrary identitypreserving transformations. The model's wiring can be learned from videos of transforming objects-or any other grouping of images into sets by their depicted object. Through a series of successively more complex empirical tests, we study the invariance/discriminability properties of this model with respect to different transformations. First, we empirically confirm theoretical predictions (from [1]) for the case of 2D affine transformations. Next, we apply the model to non-affine transformations; as expected, it performs well on face verification tasks requiring invariance to the relatively smooth transformations of 3D rotation-in-depth and changes in illumination direction. Surprisingly, it can also tolerate clutter "transformations" which map an image of a face on one background to an image of the same face on a different background. Motivated by these empirical findings, we tested the same model on face verification benchmark tasks from the computer vision literature: Labeled Faces in the Wild, PubFig [2,3,4] and a new dataset we gathered-achieving strong performance in these highly unconstrained cases as well.
This paper proposes a new approach for face 
      verification (i.e. for predicting if 2 images represent the same person or 
      not). The approach relies on an auxiliary training set containing several 
      (15 in practice: 15) images of many different persons (in practice: 450). 
      In the following, is it assumed that face images are preprocessed with low 
      level features (HOG) followed by PCA and z-score normalization, giving one 
      template per training image. At test time, the two images to be compared 
      are encoded separately by the following steps: (i) first, the normalized 
      dot product between the template of the test image and each one of the 
      templates of the k-th person is computed (ii) second, the mean value of 
      these dot products gives the k-th component of the representation of the 
      test image (iii) the two previous steps are repeated for each different 
      training persons, resulting in a 450-d representation (as the training set 
      contains 450 persons). The same process is applied independently to the 
      two images to be compared, and the final score is the normalized dot 
      product between their so-computed representations. The experimental 
      validation is done on 4 datasets (PubFig83, a new dataset introduced in 
      the paper, PubFig and LFW). Good results are obtained but no comparisons 
      with recent related work are given. In practice, the results far below 
      from recent competing approaches. Positive points 
      =============== - the paper is well written in the sense that, in 
      addition to presenting a new approach for face recognition, it tries to 
      make links with the theory of [1] and to discuss the biological 
      plausibility of the proposed solution - the paper is clear - the 
      experiments are interesting and reasonable Negative points 
      =============== - the use of auxiliary datasets and more precisely 
      the idea of representing images by the similarities with some sort of 
      prototypes has a long history in computer vision and has received an 
      increasing attention in recent years. E.g. [a] is based on training a 
      separate linear SVM classifier for every exemplar in the training set and 
      representing images by the outputs of these classifiers. More 
      specifically, the use of auxiliary face datasets for transferring some 
      knowledge about face is the central point of several recent papers (e.g. 
      [b-i]). For example, [i] noticed that faces of the same person usually 
      share their top neighbors (in the the auxiliary set). The approach is then 
      to generate a ranking order list by sorting all other faces in the dataset 
      by absolute distance and compute distance of two faces by using their 
      ranking orders. [e] proposed the "associate-predict" model which builds on 
      an extra generic identity data set, in which each identity contains 
      multiple images with large intra-personal variation. To compare two 
      faces with different pose/illumination, it ﬁrst associate one input face 
      with alike identities from the generic identity date set. The associated 
      faces are used to predict the appearance of one input face under the 
      setting of another input face. The proposed approach, even if it also 
      relies on the use of an auxiliary training set, is sufficiently different 
      from the above mentioned approaches, but the fact they are not cited nor 
      discussed raises strong concerns over the paper. - my second concern 
      is the experimental validation and more precisely the lack of comparisons 
      with previous works. As said before, several competing approaches have 
      been proposed to use auxiliary training sets for face recognition. Some of 
      them learn distance function and are therefore not strictly comparable, 
      but several use the training set to model test images, as the proposed 
      approach does, and are therefore directly comparable. Such comparisons 
      should be given. Moreover, the LFW's results page for the category 
      'Outside training data in recognition system' (category to which the paper 
      belongs) reports results up to 95% which is much better than the 87.1% 
      obtained by the paper. - Finally, the experimental validation does not 
      demonstrate if the improvement is due to the method itself (as being an 
      application of the theory of [1]) or is simply due to use of a good 
      auxiliary dataset. A simple set of experiments consisting in representing 
      test images as the normalized dot products with the whole set of test 
      images (images will be represented by a 12,500-dimensional vector, 
      possibly reduced by PCA), without doing any pooling could be useful. If 
      the results are as good as the proposed approach, it would support more 
      the theory of [a] than the theory of [1]. A the end, I have not been able 
      to determine if the proposed face recognition framework is actually an 
      application of [1] or more an exemplar based recognition system [a] (the 
      SVM of [a] being replaced by a simple dot product). Another way to say is 
      to say that there is a big gap between sections 1,2 and 3 where the notion 
      of orbit, geometric transforms, groups, CFD signatures, etc. are at the 
      basis of the recognition process and sections 4 and 5 where there is no 
      more transforms, CDF signatures or whatever but a simple exemplar based 
      recognition approach made by simple dot products. 
      References [a] Tomasz Malisiewicz, Abhinav Gupta, 
      Alexei A. Efros. Ensemble of Exemplar-SVMs for Object Detection and Beyond 
      . In ICCV, 2011 [b] Neeraj Kumar, Alexander C. Berg, Peter N. 
      Belhumeur, and Shree K. Nayar. Attribute and Simile Classifiers for Face 
      Verification. International Conference on Computer Vision (ICCV), 
      2009. [c] Vinod Nair and Geoffrey E. Hinton. Rectified Linear 
      Units Improve Restricted Boltzmann Machines. International Conference on 
      Machine Learning (ICML), 2010. [d] Zhimin Cao, Qi Yin, Xiaoou 
      Tang, and Jian Sun. Face Recognition with Learning-based Descriptor. 
      Computer Vision and Pattern Recognition (CVPR), 2010. [e] Qi Yin, 
      Xiaoou Tang, and Jian Sun. An Associate-Predict Model for Face 
      Recognition. Computer Vision and Pattern Recognition (CVPR), 2011. 
      [f] Thomas Berg and Peter N. Belhumeur. Tom-vs-Pete Classifiers 
      and Identity-Preserving Alignment for Face Verification. British Machine 
      Vision Conference (BMVC), 2012. [g] Dong Chen, Xudong Cao, Liwei 
      Wang, Fang Wen, and Jian Sun. Bayesian Face Revisited: A Joint 
      Formulation. European Conference on Computer Vision (ECCV), 2012. 
      [h] Dong Chen, Xudong Cao, Fang Wen, and Jian Sun. Blessing of 
      Dimensionality: High-dimensional Feature and Its Efficient Compression for 
      Face Verification. Computer Vision and Pattern Recognition (CVPR), 2013. 
      [i] Chunhui Zhu, A Rank-Order Distance based Clustering Algorithm 
      for Face Tagging, Fang Wen Jian Sun. Computer Vision and Pattern 
      Recognition (CVPR), 2011.