Multi-View Perceptron: a Deep Model for Learning Face Identity and View Representations
Various factors, such as identity, view, and illumination, are coupled in face images. Disentangling the identity and view representations is a major challenge in face recognition. Existing face recognition systems either use handcrafted features or learn features discriminatively to improve recognition accuracy. This is different from the behavior of primate brain. Recent studies [5,19] discovered that primate brain has a face-processing network, where view and identity are processed by different neurons. Taking into account this instinct, this paper proposes a novel deep neural net, named multi-view perceptron (MVP), which can untangle the identity and view features, and in the meanwhile infer a full spectrum of multi-view images, given a single 2D face image. The identity features of MVP achieve superior performance on the MultiPIE dataset. MVP is also capable to interpolate and predict images under viewpoints that are unobserved in the training data.
This paper presents a novel way to do multi-view representation of faces using a neural-net like model with deterministic and probabilistic units. The network is trained to recover the identity and view. It can be used to synthesize a new view as well. Generally, the paper does a good job at disentangling the contributing factors to the success of the method, and the results seem quite good.I wish the authors presented more evidence for why they chose to design their model they way they did. For one, it is not clear that a non-deterministic approach is really needed in this case -- could they have done a model that is fully feed-forward with deterministic neurons? It would have certainly avoided the need to variational/importance sampling inference.Quality: generally, a good paper with an interesting and well-grounded model, evaluated well on a competitive benchmark and Clarity: The paper is well-written and easy to read, for most of the part.Originality: in some sense, the approach is comparable to the work of Geoff Hinton and collaborators, who use stacks of Restricted Boltzmann Machines, where the top RBM can have the label as a sideSignificance: this is likely to be of interest to the facial recognition community, but also to other researchers who work on problems where a hybrid deterministic and generative approach may be a good solution.Comments and questions:- How did the authors come up with the structure in Figure 2? What are the insights behind the design choices made?- The authors should define q(h) at line 165- Is the MCEM method crucial to the model presented? More analysis on why this particular optimization method was used is in order since MCEM is not exactly a widely used method in the community.- For inference: how big is S at line 235? How expensive is inference generally? Some numbers and O() notation results would be good.- Is LDA necessary to get the best results? How do the results look like without LDA?- I don't think the authors should be using the tangent with how the brain works, except very sparsely. There is not much evidence that the presented method is particularly close to how humans do facial recognition or view point interpolation. - Did the authors try their approach on the Labeled Faces in the Wild benchmark? If yes, do they have results to report?