Answers.Body: <p>The problem originated because of the nature of the code. </p>
  
  <p>Code:
  <a href="https://github.com/AISangam/Facenet-Real-time-face-recognition-using-deep-learning-Tensorflow/blob/master/classifier.py" rel="nofollow noreferrer">https://github.com/AISangam/Facenet-Real-time-face-recognition-using-deep-learning-Tensorflow/blob/master/classifier.py</a></p>
  
  
  
  <pre><code>                model = SVC(kernel='linear', probability=True)
                  model.fit(emb_array, label)
  
                  class_names = [cls.name.replace('_', ' ') for cls in img_data]
  </code></pre>
  
  <p>As you see the code uses a SVC (Support Vector Classifier) to classify the classes. The SVC (or SVM) does not have an extra class for <code>unknown</code> class.</p>
  
  <p>For the <code>threshold</code> variable, it is used in face detection, aka drawing a bounding box around the face for FaceNet to classify it. </p>
  
  <p>Code:</p>
  
  <p><a href="https://github.com/AISangam/Facenet-Real-time-face-recognition-using-deep-learning-Tensorflow/blob/master/identify_face_image.py" rel="nofollow noreferrer">https://github.com/AISangam/Facenet-Real-time-face-recognition-using-deep-learning-Tensorflow/blob/master/identify_face_image.py</a></p>
  
  
  
  <pre><code>            frame = frame[:, :, 0:3]
              bounding_boxes, _ = detect_face.detect_face(frame, minsize, pnet, rnet, onet, threshold, factor)
              nrof_faces = bounding_boxes.shape[0]
  </code></pre>
  
  <p>As you can see, the <code>threshold</code> variable is only used in detecting the bounding box.</p>
  
  <p>Code for getting class name:</p>
  
  
  
  <pre><code>
  predictions = model.predict_proba(emb_array)
                      print(predictions)
                      best_class_indices = np.argmax(predictions, axis=1)
                      # print(best_class_indices)
                      best_class_probabilities = predictions[np.arange(len(best_class_indices)), best_class_indices]
                      print(best_class_probabilities)
                      cv2.rectangle(frame, (bb[i][0], bb[i][1]), (bb[i][2], bb[i][3]), (0, 255, 0), 2)    #boxing face
  
                      #plot result idx under box
                      text_x = bb[i][0]
                      text_y = bb[i][3] + 20
                      print('Result Indices: ', best_class_indices[0])
                      print(HumanNames)
  </code></pre>
  
  <p>You can see that no <code>unknown</code> class can be found. </p>
  
  <h1>Solution</h1>
  
  <p>You can try adding another threshold value and check if the <code>predictions</code> maximum value is lower than the threshold value. I have little experience in tensor flow so this is just a proof of concept, not sure if it will work.</p>
  
  <pre><code>best_class_probabilities = predictions[np.arange(len(best_class_indices)), best_class_indices] #original code
  if(best_class_probabilities &lt; threshold_2):
      best_class_indices = -1
  HumanNames = "unknown"
  </code></pre>
  
  <p>By the way, because of the nature of triplet loss, you don't have to add and extra class to the SVC/SVM as the embedding model is locked and not trained, so unknown class embeddings will be very different to the known class. However you can try either approach.</p>
  
  <p>Hope it can help you can have a nice day!</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-11T13:13:33.210Z
Answers.Text: for classifying as unknown i have to put the threshold 89% since for unknown persons sometimes best_class_probabilities is 89% ! .But how a unknown person can be as close as 89% ! .That is why i asked you how Facenet Model works .I guess in program there should be following feature - Suppose there are 4 people in training data set , then array of probability should be of size 5 , last element of array should be probability that person is not known !Whereas first 4 element representing the probability that the person is one of the person in data set. It is possible to do that in Facenet Model ?
Answers.UserDisplayName: TheReal__Mike
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-11T13:14:03.443Z
Answers.Text: BTW clement Hui you helped me a lot and it seems that you are very  passionate in AI
Answers.UserDisplayName: TheReal__Mike
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-11T14:14:48.200Z
Answers.Text: @Mike for the problem you describe, there is two ways of face recognition using facenet embeddings. The first way is the way used in the github repo. It simply uses a SVM to output the classified class. The other method is to save the embeddings of the photos of people in the dataset, and compare them with the image embedding to classify by calculating the distance. The minimum distance in all classes is taken and if the distance is below a certain threshold, the class is taken as answer. Otherwise, the class will be unknown.
Answers.UserDisplayName: Clement
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-11T14:16:24.213Z
Answers.Text: However this method have it's disadvantages as with a lot of people in it's database, it takes a long time to loop through all embeddings. However this method does not require training and no additional training is required if you add a new class.
Answers.UserDisplayName: Clement
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-11T14:17:10.010Z
Answers.Text: See this webpage for details: https://medium.com/intro-to-artificial-intelligence/one-shot-learning-explained-using-facenet-dff5ad52bd38
Answers.UserDisplayName: Clement
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-11T14:17:37.050Z
Answers.Text: Btw @Mike thank you very much for your appreciation. Hope I can help you.
Answers.UserDisplayName: Clement
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-11T14:20:55.410Z
Answers.Text: also the comment is also the answer to your question. I will add an answer asap.
Answers.UserDisplayName: Clement
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-11T10:21:57.777Z
Answers.LastActivityDate: 2019-11-11T10:21:57.777Z
Answers.UserDisplayName: Clement
Body: <p>Suppose i trained the images of two people say Bob , Thomas .When i run the algorithm to detect the face of a totally different person from these two say John , then John is recognized as Bob or Thomas.How to avoid this ?</p>
  
  <p>I am studying a face recognition model on GitHub(<a href="https://github.com/AISangam/Facenet-Real-time-face-recognition-using-deep-learning-Tensorflow" rel="nofollow noreferrer">link</a>) which uses Facenet model. Problem is when an unknown image (the image which is not in training data set) is given to identify , it identifies the unknown person as one the person in the data set .I searched on web and i found i need to increase threshold value .I guess i need to increase the <a href="https://github.com/AISangam/Facenet-Real-time-face-recognition-using-deep-learning-Tensorflow/blob/b351d5d3f4766572868236eed78a25082da44c27/identify_face_image.py#L28" rel="nofollow noreferrer">threshold</a>. But when i am increasing the threshold value to 0.99,0.99,99 then only it is rejecting the unknown image (image of the person who is not in data set) and sometimes even rejecting the image of person who is in dataset.</p>
  
  <p>I guess by increasing the threshold value what we are assuring is that an image is classified as one of the person in training data only when they are close enough.</p>
  
  <p>How to make changes so that the model works properly ?And can someone explain Threshold in Facenet model better.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-11-11T08:52:59.280Z
Comments.Text: which dataset are you using? Also, a threshold of 0.99 maybe too large, you can try 0.8 or something like that? Thanks @Mike
Comments.UserDisplayName: Clement
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-11-11T09:00:16.997Z
Comments.Text: @ClementHui When i reduce to 0.8 then it classifies the image as one of the images in data set .(i.e John is recognized as bob or thomas)
Comments.UserDisplayName: TheReal__Mike
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-11-11T09:01:47.233Z
Comments.Text: What is your dataset? How large is it? The blog page suggested to have at least 40 images of each cladd
Comments.UserDisplayName: Clement
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-11-11T09:03:05.673Z
Comments.Text: @ClementHui My data set consist of 5 people ,  each with 1000 images .It gives good accuracy when unknown person is similar to one of the 5 person but if suppose i try to detect image of some person different from those which are in data set then also it detects them as one of them. What i want is it should classify that person as unknown !
Comments.UserDisplayName: TheReal__Mike
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-11-11T10:22:11.480Z
Comments.Text: added answer. Hope it helps! @Mike
Comments.UserDisplayName: Clement
Tags: <neural-networks><machine-learning>
Title: Three step threshold in Facenet model of face recogniton
UserDisplayName: TheReal__Mike
ThreadText: # Three step threshold in Facenet model of face recogniton
  
  (Asked by: username_0 on 2019-10-31)
  
  Suppose i trained the images of two people say Bob , Thomas .When i run the algorithm to detect the face of a totally different person from these two say John , then John is recognized as Bob or Thomas.How to avoid this ?
  
  I am studying a face recognition model on GitHub([link](https://github.com/AISangam/Facenet-Real-time-face-recognition-using-deep-learning-Tensorflow)) which uses Facenet model. Problem is when an unknown image (the image which is not in training data set) is given to identify , it identifies the unknown person as one the person in the data set .I searched on web and i found i need to increase threshold value .I guess i need to increase the [threshold](https://github.com/AISangam/Facenet-Real-time-face-recognition-using-deep-learning-Tensorflow/blob/b351d5d3f4766572868236eed78a25082da44c27/identify_face_image.py#L28). But when i am increasing the threshold value to 0.99,0.99,99 then only it is rejecting the unknown image (image of the person who is not in data set) and sometimes even rejecting the image of person who is in dataset.
  
  I guess by increasing the threshold value what we are assuring is that an image is classified as one of the person in training data only when they are close enough.
  
  How to make changes so that the model works properly ?And can someone explain Threshold in Facenet model better.
  
  **Tags:** neural-networks, machine-learning
  > **Comments:**
  > **username_1**: which dataset are you using? Also, a threshold of 0.99 maybe too large, you can try 0.8 or something like that? Thanks @Mike  
  > **username_0**: @ClementHui When i reduce to 0.8 then it classifies the image as one of the images in data set .(i.e John is recognized as bob or thomas)  
  > **username_1**: What is your dataset? How large is it? The blog page suggested to have at least 40 images of each cladd  
  > **username_0**: @ClementHui My data set consist of 5 people ,  each with 1000 images .It gives good accuracy when unknown person is similar to one of the 5 person but if suppose i try to detect image of some person different from those which are in data set then also it detects them as one of them. What i want is it should classify that person as unknown !  
  > **username_1**: added answer. Hope it helps! @Mike  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-11-11. Score: 2 | Accepted answer
  
  The problem originated because of the nature of the code. 
  Code:
  https://github.com/AISangam/Facenet-Real-time-face-recognition-using-deep-learning-Tensorflow/blob/master/classifier.py
                  model = SVC(kernel='linear', probability=True)
                  model.fit(emb_array, label)
  
                  class_names = [cls.name.replace('_', ' ') for cls in img_data]
  
  As you see the code uses a SVC (Support Vector Classifier) to classify the classes. The SVC (or SVM) does not have an extra class for unknown class.
  For the threshold variable, it is used in face detection, aka drawing a bounding box around the face for FaceNet to classify it. 
  Code:
  https://github.com/AISangam/Facenet-Real-time-face-recognition-using-deep-learning-Tensorflow/blob/master/identify_face_image.py
              frame = frame[:, :, 0:3]
              bounding_boxes, _ = detect_face.detect_face(frame, minsize, pnet, rnet, onet, threshold, factor)
              nrof_faces = bounding_boxes.shape[0]
  
  As you can see, the threshold variable is only used in detecting the bounding box.
  Code for getting class name:
  
  predictions = model.predict_proba(emb_array)
                      print(predictions)
                      best_class_indices = np.argmax(predictions, axis=1)
                      # print(best_class_indices)
                      best_class_probabilities = predictions[np.arange(len(best_class_indices)), best_class_indices]
                      print(best_class_probabilities)
                      cv2.rectangle(frame, (bb[i][0], bb[i][1]), (bb[i][2], bb[i][3]), (0, 255, 0), 2)    #boxing face
  
                      #plot result idx under box
                      text_x = bb[i][0]
                      text_y = bb[i][3] + 20
                      print('Result Indices: ', best_class_indices[0])
                      print(HumanNames)
  
  You can see that no unknown class can be found. 
  Solution
  You can try adding another threshold value and check if the predictions maximum value is lower than the threshold value. I have little experience in tensor flow so this is just a proof of concept, not sure if it will work.
  best_class_probabilities = predictions[np.arange(len(best_class_indices)), best_class_indices] #original code
  if(best_class_probabilities < threshold_2):
      best_class_indices = -1
  HumanNames = "unknown"
  
  By the way, because of the nature of triplet loss, you don't have to add and extra class to the SVC/SVM as the embedding model is locked and not trained, so unknown class embeddings will be very different to the known class. However you can try either approach.
  Hope it can help you can have a nice day!
  
  > **Comments:**
  > **username_0**: for classifying as unknown i have to put the threshold 89% since for unknown persons sometimes best_class_probabilities is 89% ! .But how a unknown person can be as close as 89% ! .That is why i asked you how Facenet Model works .I guess in program there should be following feature - Suppose there are 4 people in training data set , then array of probability should be of size 5 , last element of array should be probability that person is not known !Whereas first 4 element representing the probability that the person is one of the person in data set. It is possible to do that in Facenet Model ?  
  > **username_0**: BTW clement Hui you helped me a lot and it seems that you are very  passionate in AI  
  > **username_1**: @Mike for the problem you describe, there is two ways of face recognition using facenet embeddings. The first way is the way used in the github repo. It simply uses a SVM to output the classified class. The other method is to save the embeddings of the photos of people in the dataset, and compare them with the image embedding to classify by calculating the distance. The minimum distance in all classes is taken and if the distance is below a certain threshold, the class is taken as answer. Otherwise, the class will be unknown.  
  > **username_1**: However this method have it's disadvantages as with a lot of people in it's database, it takes a long time to loop through all embeddings. However this method does not require training and no additional training is required if you add a new class.  
  > **username_1**: See this webpage for details: https://medium.com/intro-to-artificial-intelligence/one-shot-learning-explained-using-facenet-dff5ad52bd38  
  > **username_1**: Btw @Mike thank you very much for your appreciation. Hope I can help you.  
  > **username_1**: also the comment is also the answer to your question. I will add an answer asap.  
  
  ---
  
  