Answers.Body: <p>As Neil Slater said it depends on how the model was trained. Now if you go to the FaceNet implementation in TF in github you can see that in the face alignment they do <a href="https://github.com/davidsandberg/facenet/blob/096ed770f163957c1e56efa7feeb194773920f6e/src/align/align_dataset_mtcnn.py#L124" rel="nofollow noreferrer">resize without aspect ratio keeping</a> (old <code>scipy.misc.imresize</code> does not keep aspect ratio) so if you are using this implementation the answer to your question is: <em>it does not affect the accuracy</em>.</p>
  <p>But if we want to elaborate the golden rule is: <strong>not to change the face aspect ratio</strong> because that is causing a distortion of the input data for the embedding computation.</p>
  <p>In most face recognition tasks once you detect a bounding box you do a resize with aspect ratio keeping. You can accomplish that with several libraries:</p>
  <ul>
  <li><a href="https://pillow.readthedocs.io/en/stable/reference/Image.html?highlight=thumbnail#PIL.Image.Image.thumbnail" rel="nofollow noreferrer">Pillow</a> Thumbnail method</li>
  <li><a href="https://albumentations.ai/docs/api_reference/augmentations/geometric/resize/#albumentations.augmentations.geometric.resize.LongestMaxSize" rel="nofollow noreferrer">Albumentations</a> LongestMaxSize method</li>
  </ul>
  <p>I would recommend to use resize with aspect ratio keeping.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-04-20T13:31:08.207Z
Answers.LastActivityDate: 2022-04-20T13:31:08.207Z
Answers.UserDisplayName: JVGD
Body: <p><a href="https://i.sstatic.net/s7pTl.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/s7pTl.png" alt="enter image description here" /></a></p>
  <p>As far as I know,</p>
  <ol>
  <li><p>FaceNet requires a square image as an input.</p>
  </li>
  <li><p>MTCNN can detect and crop the original image as a square, but distortion occurs.</p>
  </li>
  </ol>
  <p>Is it okay to feed the converted (now square) distorted image into FaceNet? Does it affect the accuracy of calculation similarity (embedding)?</p>
  <p>For similarity (classification of known faces), I am going to put some custom layers upon FaceNet.</p>
  <p>(If it's okay, maybe because every other image would be distorted no matter what? So, it would not compare normal image vs distorted image, but distorted image vs distorted image, which would be fair?)</p>
  <p>Original issue: <a href="https://github.com/timesler/facenet-pytorch/issues/181" rel="nofollow noreferrer">https://github.com/timesler/facenet-pytorch/issues/181</a>.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-10-23T09:03:52.383Z
Comments.Text: You don't have to crop out the detection box as-is, you could crop out a square centred on the box. Have you tried that for comparison?
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-10-24T04:13:37.623Z
Comments.Text: @NeilSlater Yeah right, but the problem is, if I do so, unnecessary portions like hair will be included. For example, in general, a human's face is vertically long. If I crop the image in a square without distortion, then it's very likely the hair would be included. But a same person(especially woman) can have long or short hair depending on images. And I wonder if the calculation of embedding with facenet would be affected by such a thing. Actually, facenet expects "only face" in a square format as an input, and I've seen examples of distortion to make face a square. Would it be okay?
Comments.UserDisplayName: jjangga
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-10-24T07:43:20.853Z
Comments.Text: I don't know if it is OK or not, it depends on what inputs the model was trained on. You either need advice from someone who has used the specific model that you are using, or you need to try the different approaches that you are concerned about and measure their accuracy. There is no general rule covering all image classifiers.
Comments.UserDisplayName: Neil Slater
Tags: <data-preprocessing><face-recognition><face-detection><facenet>
Title: Does the converted (now square) distorted image of a face affect the accuracy of the calculation of the similarity in FaceNet?
UserDisplayName: jjangga
ThreadText: # Does the converted (now square) distorted image of a face affect the accuracy of the calculation of the similarity in FaceNet?
  
  (Asked by: username_0 on 2021-10-22)
  
  [![enter image description here](https://i.sstatic.net/s7pTl.png)](https://i.sstatic.net/s7pTl.png)
  
  As far as I know,
  
  1. FaceNet requires a square image as an input.
  2. MTCNN can detect and crop the original image as a square, but distortion occurs.
  
  Is it okay to feed the converted (now square) distorted image into FaceNet? Does it affect the accuracy of calculation similarity (embedding)?
  
  For similarity (classification of known faces), I am going to put some custom layers upon FaceNet.
  
  (If it's okay, maybe because every other image would be distorted no matter what? So, it would not compare normal image vs distorted image, but distorted image vs distorted image, which would be fair?)
  
  Original issue: <https://github.com/timesler/facenet-pytorch/issues/181>.
  
  **Tags:** data-preprocessing, face-recognition, face-detection, facenet
  
  > **Comments:**
  > **username_1**: You don't have to crop out the detection box as-is, you could crop out a square centred on the box. Have you tried that for comparison?  
  > **username_0**: @username_1 Yeah right, but the problem is, if I do so, unnecessary portions like hair will be included. For example, in general, a human's face is vertically long. If I crop the image in a square without distortion, then it's very likely the hair would be included. But a same person(especially woman) can have long or short hair depending on images. And I wonder if the calculation of embedding with facenet would be affected by such a thing. Actually, facenet expects "only face" in a square format as an input, and I've seen examples of distortion to make face a square. Would it be okay?  
  > **username_1**: I don't know if it is OK or not, it depends on what inputs the model was trained on. You either need advice from someone who has used the specific model that you are using, or you need to try the different approaches that you are concerned about and measure their accuracy. There is no general rule covering all image classifiers.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2022-04-20. Score: 1
  
  As username_1 username_1 said it depends on how the model was trained. Now if you go to the FaceNet implementation in TF in github you can see that in the face alignment they do [resize without aspect ratio keeping](https://github.com/davidsandberg/facenet/blob/096ed770f163957c1e56efa7feeb194773920f6e/src/align/align_dataset_mtcnn.py#L124) (old `scipy.misc.imresize` does not keep aspect ratio) so if you are using this implementation the answer to your question is: *it does not affect the accuracy*.
  
  But if we want to elaborate the golden rule is: **not to change the face aspect ratio** because that is causing a distortion of the input data for the embedding computation.
  
  In most face recognition tasks once you detect a bounding box you do a resize with aspect ratio keeping. You can accomplish that with several libraries:
  
  * [Pillow](https://pillow.readthedocs.io/en/stable/reference/Image.html?highlight=thumbnail#PIL.Image.Image.thumbnail) Thumbnail method
  * [Albumentations](https://albumentations.ai/docs/api_reference/augmentations/geometric/resize/#albumentations.augmentations.geometric.resize.LongestMaxSize) LongestMaxSize method
  
  I would recommend to use resize with aspect ratio keeping.
  
  
  ---
  
  