Answers.Body: <p>Landmark retrieval has photographs of landmarks that you need to find out. Consider the degrees of freedom for this, the landmarks can many different colours (more than humans' faces) and also the colour range is all over the place (a landmark may be blue or white or red). The shapes of the various landmarks will also vary.</p>
  <p>Now, consider face recognition problem. All humans faces look alike morphologically. If you look at the colour, it is not as varied as landmark recognition.</p>
  <p>Because of the inherent data in both the problems, the research focuses on the diverging lines of thought. Rich descriptors are good for landmarks because the data itself is very rich and mired with variation. On the other hand, discriminative features are more desirable for face recognition because faces are more similar and less rich in variation, so differentiating between is hard.</p>
  <p>It is the requirement of the problem that steers research in diverging directions.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-08T21:18:06.447Z
Answers.Text: Thanks, that makes sense. May I ask, are you part of either of these research communities? Your answer is good, but I just want to know if I've heard it from the horse's mouth
Answers.UserDisplayName: Alexander Soare
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-09T08:38:21.877Z
Answers.Text: Yes, I do create and research facial recognition systems on a billion-scale.
Answers.UserDisplayName: Abhishek Verma
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-09T09:47:17.783Z
Answers.Text: Great, thank you. I'll leave this open for a bit and see if anyone else has something to say, then I'll close it once the bounty is rewarded.
Answers.UserDisplayName: Alexander Soare
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-03-08T18:57:37.960Z
Answers.LastActivityDate: 2021-03-08T18:57:37.960Z
Answers.UserDisplayName: Abhishek Verma
Body: <h2>Context and detail</h2>
  <p>I've been working on a particular image retrieval problem and I've found two popular threads in the literature:</p>
  <p><strong>Image retrieval</strong> (usually benchmarked with landmark retrieval datasets)</p>
  <ul>
  <li>[x] <a href="https://arxiv.org/abs/1404.1777" rel="nofollow noreferrer">Neural codes for Image Retrieval</a></li>
  <li>[x] <a href="https://arxiv.org/abs/1511.05879" rel="nofollow noreferrer">Particular object retrieval with integral max-pooling of CNN activations</a></li>
  <li>[ ] <a href="https://arxiv.org/pdf/1604.01325.pdf" rel="nofollow noreferrer">Deep Image Retrieval: Learning global representations for image search</a></li>
  <li>[ ] <a href="https://arxiv.org/pdf/1610.07940.pdf" rel="nofollow noreferrer">End-to-end Learning of Deep Visual Representations for Image Retrieval</a></li>
  <li>[ ] <a href="https://arxiv.org/abs/1612.06321" rel="nofollow noreferrer">Large-Scale Image Retrieval with Attentive Deep Local Features</a></li>
  <li>[ ] <a href="https://arxiv.org/pdf/1711.02512.pdf" rel="nofollow noreferrer">Fine-tuning CNN Image Retrieval with No Human Annotation</a></li>
  </ul>
  <p><strong>Face recognition/verification</strong>:</p>
  <ul>
  <li>[x] <a href="https://arxiv.org/abs/1503.03832" rel="nofollow noreferrer">Facenet</a></li>
  <li>[x] <a href="https://www.robots.ox.ac.uk/%7Evgg/publications/2015/Parkhi15/parkhi15.pdf" rel="nofollow noreferrer">Deep Face Recognition</a></li>
  <li>[x] <a href="https://arxiv.org/pdf/1704.08063.pdf" rel="nofollow noreferrer">SphereFace</a></li>
  <li>[ ] <a href="https://arxiv.org/pdf/1801.07698.pdf" rel="nofollow noreferrer">Arcface</a></li>
  </ul>
  <p>I'm still making my way through these lists and more (I've checked the ones I've looked at already) but I'm starting to get a sense that there's not much overlap in the techniques used, or the collective trains of thought in the research community. Here are the main points of divergence where I think both communities should be borrowing from each other.</p>
  <ul>
  <li>Facial recognition seems to focus on getting embeddings to be as discriminative as possible by playing around with loss functions and training methods, whereas image retrieval seems to care more about ways of extracting feature descriptors from CNN pretrained backbones (types of pooling operations, which feature maps to look at, etc..).</li>
  <li>Image retrieval has a considerable amount of work on what needs to happen after an embedding is obtained. Eg: dimensionality reduction, whitening + l2 norm, databise-side augmentation, query expansion, reranking etc</li>
  <li>Facial recognition cares about keeping a minimum margin between non-matching faces in order to avoid mismatches, but I would think that should be imposed in image retrieval tasks as well (this is kind of a sub-point to my first point)</li>
  </ul>
  <p><strong>So to sum up</strong>: Why is it that facial recognition focuses on generating discriminative embeddings, while landmark retrieval focusses on generating rich &quot;descriptors&quot;? Why does landmark retrieval use this cool bag of tricks for database search while facial recognition just mentions kNN? Shouldn't all these considerations boost performance in either domain?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-03-08T12:43:54.643Z
Comments.Text: I have used opencv facial recogition and convolution neural network facial recognition.   Are you interested in either?
Comments.UserDisplayName: Golden Lion
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-03-08T13:12:34.187Z
Comments.Text: Interested in deep learning approaches
Comments.UserDisplayName: Alexander Soare
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-03-08T13:18:49.800Z
Comments.Text: Keras cnn ?   what is your input shape of the image
Comments.UserDisplayName: Golden Lion
Tags: <convolutional-neural-networks><facial-recognition><content-based-image-retrieval>
Title: Why are the landmark retrieval and facial recognition literature so divergent?
UserDisplayName: Alexander Soare
ThreadText: # Why are the landmark retrieval and facial recognition literature so divergent?
  
  (Asked by: username_0 on 2021-03-03)
  
  ## Context and detail
  
  I've been working on a particular image retrieval problem and I've found two popular threads in the literature:
  
  **Image retrieval** (usually benchmarked with landmark retrieval datasets)
  
  * [x] [Neural codes for Image Retrieval](https://arxiv.org/abs/1404.1777)
  * [x] [Particular object retrieval with integral max-pooling of CNN activations](https://arxiv.org/abs/1511.05879)
  * [ ] [Deep Image Retrieval: Learning global representations for image search](https://arxiv.org/pdf/1604.01325.pdf)
  * [ ] [End-to-end Learning of Deep Visual Representations for Image Retrieval](https://arxiv.org/pdf/1610.07940.pdf)
  * [ ] [Large-Scale Image Retrieval with Attentive Deep Local Features](https://arxiv.org/abs/1612.06321)
  * [ ] [Fine-tuning CNN Image Retrieval with No Human Annotation](https://arxiv.org/pdf/1711.02512.pdf)
  
  **Face recognition/verification**:
  
  * [x] [Facenet](https://arxiv.org/abs/1503.03832)
  * [x] [Deep Face Recognition](https://www.robots.ox.ac.uk/%7Evgg/publications/2015/Parkhi15/parkhi15.pdf)
  * [x] [SphereFace](https://arxiv.org/pdf/1704.08063.pdf)
  * [ ] [Arcface](https://arxiv.org/pdf/1801.07698.pdf)
  
  I'm still making my way through these lists and more (I've checked the ones I've looked at already) but I'm starting to get a sense that there's not much overlap in the techniques used, or the collective trains of thought in the research community. Here are the main points of divergence where I think both communities should be borrowing from each other.
  
  * Facial recognition seems to focus on getting embeddings to be as discriminative as possible by playing around with loss functions and training methods, whereas image retrieval seems to care more about ways of extracting feature descriptors from CNN pretrained backbones (types of pooling operations, which feature maps to look at, etc..).
  * Image retrieval has a considerable amount of work on what needs to happen after an embedding is obtained. Eg: dimensionality reduction, whitening + l2 norm, databise-side augmentation, query expansion, reranking etc
  * Facial recognition cares about keeping a minimum margin between non-matching faces in order to avoid mismatches, but I would think that should be imposed in image retrieval tasks as well (this is kind of a sub-point to my first point)
  
  **So to sum up**: Why is it that facial recognition focuses on generating discriminative embeddings, while landmark retrieval focusses on generating rich "descriptors"? Why does landmark retrieval use this cool bag of tricks for database search while facial recognition just mentions kNN? Shouldn't all these considerations boost performance in either domain?
  
  **Tags:** convolutional-neural-networks, facial-recognition, content-based-image-retrieval
  
  > **Comments:**
  > **username_1**: I have used opencv facial recogition and convolution neural network facial recognition.   Are you interested in either?  
  > **username_0**: Interested in deep learning approaches  
  > **username_1**: Keras cnn ?   what is your input shape of the image  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2021-03-08. Score: 2 | Accepted answer
  
  Landmark retrieval has photographs of landmarks that you need to find out. Consider the degrees of freedom for this, the landmarks can many different colours (more than humans' faces) and also the colour range is all over the place (a landmark may be blue or white or red). The shapes of the various landmarks will also vary.
  
  Now, consider face recognition problem. All humans faces look alike morphologically. If you look at the colour, it is not as varied as landmark recognition.
  
  Because of the inherent data in both the problems, the research focuses on the diverging lines of thought. Rich descriptors are good for landmarks because the data itself is very rich and mired with variation. On the other hand, discriminative features are more desirable for face recognition because faces are more similar and less rich in variation, so differentiating between is hard.
  
  It is the requirement of the problem that steers research in diverging directions.
  
  > **Comments:**
  > **username_0**: Thanks, that makes sense. May I ask, are you part of either of these research communities? Your answer is good, but I just want to know if I've heard it from the horse's mouth  
  > **username_2**: Yes, I do create and research facial recognition systems on a billion-scale.  
  > **username_0**: Great, thank you. I'll leave this open for a bit and see if anyone else has something to say, then I'll close it once the bounty is rewarded.  
  
  ---
  
  