Deep Learning Face Representation by Joint Identification-Verification
The key challenge of face recognition is to develop effective feature representations for reducing intra-personal variations while enlarging inter-personal differences. In this paper, we show that it can be well solved with deep learning and using both face identification and verification signals as supervision. The Deep IDentification-verification features (DeepID2) are learned with carefully designed deep convolutional networks. The face identification task increases the inter-personal variations by drawing DeepID2 features extracted from different identities apart, while the face verification task reduces the intra-personal variations by pulling DeepID2 features extracted from the same identity together, both of which are essential to face recognition. The learned DeepID2 features can be well generalized to new identities unseen in the training data. On the challenging LFW dataset [11], 99.15% face verification accuracy is achieved. Compared with the best previous deep learning result [20] on LFW, the error rate has been significantly reduced by 67%.
This paper's core finding is that combining an identity classification task as well as metric-learning-style verification task helps to learn better features for face classification/verification.  The "verification task" here tries to decrease feature-space distance between instances of the same identity, and increase distance between those of different identities.  This improvement is embedded in a state-of-the-art system for face verification, which uses convnets trained on many (400) different views to generate features, distilled into a small set of 25 using feature selection.  Very good results are obtained and experiments performed using LFW as a test set.Overall, these are very good results obtained using a somewhat complex pipeline, and a good investigation into the contribution of each "task" in the loss for feature learning.  Combining the tasks has been hinted at before (e.g. siamese network after classification pre-training for DeepFace), but this work brings this out specifically.  Some of the writing could be a bit less dry (and edited for English), but this does not interfere with the understanding of the paper.Further comments:* I don't understand how "m" in eqn 1 was selected.  If it is learned as a parameter, it seems it would collapse to m=0, or at most the minimum ||fi-fj|| (even with the procedure indicated at l.152), since this would always give zero Verif error for negative pairs.* Is there a more concrete reason why including the identification task improves verification performance and/or learning ("richer information" is a bit vague)?  E.g. the gradient from the verification task pushes apart just two points, while the softmax grad in the identification task pushes the correct identity away from the classification-layer templates for *all* other identities at the same time.* It would be nice to see individual performance of the best few selected network regions/views, or an evaluation using 1, 2, 4, ..., 25 views.  The final combination achieves good performance, but it would be good to also see the individual contributions.* l.38 "eternal topic":  this is an odd choice of word (it implies the topic will be debated forever, when in fact it seems to be making good progress); a "central topic" may be better here.