Answers.Body: <blockquote>
  <p>Can I detect unique people in a video?</p>
  </blockquote>
  <p>Yes. However, for some applications commercial systems are not good enough: <a href="https://netzpolitik.org/2017/ueberwachungslabor-berlin-suedkreuz-tracking-und-gesichtserkennung-geplant/" rel="nofollow noreferrer">German source</a></p>
  <p>The concepts you are looking for are:</p>
  <ul>
  <li><a href="https://en.m.wikipedia.org/wiki/Optical_flow" rel="nofollow noreferrer">Optical Flow</a>: for two frames, define for each point how it moved</li>
  <li>Object tracking: given a single object in one frame, find it in the next. Optical flow helps with that.</li>
  <li><a href="https://www.pyimagesearch.com/2018/11/26/instance-segmentation-with-opencv/" rel="nofollow noreferrer">Object detection</a>: look at YOLO / SSD. Small introduction <a href="https://medium.com/comet-app/review-of-deep-learning-algorithms-for-object-detection-c1f3d437b852" rel="nofollow noreferrer">here</a>.</li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-11-30T07:42:05.667Z
Answers.Text: Thanks Martin, is object tracking robust if the object in interest are coming in and out of the frames. Optical flow will just track it if it is continuously in the stream.
Answers.UserDisplayName: vc_dim
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-11-30T07:54:30.457Z
Answers.Text: Correct. But Optical flow gives you a good idea where to look for new objects.
Answers.UserDisplayName: Martin Thoma
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-11-30T07:59:10.813Z
Answers.Text: Makes sense, Let me try that. Thanks a lot Martin.
Answers.UserDisplayName: vc_dim
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-11-30T06:41:15.710Z
Answers.LastActivityDate: 2021-01-18T12:55:13.160Z
Answers.LastEditDate: 2021-01-18T12:55:13.160Z
Answers.UserDisplayName: Martin Thoma
Answers.Body: <p>You can detect faces in video frames by employing the standard method of Viola-Jones detector (though there are sophisticated ones available now). You can get the paper from <a href="https://ieeexplore.ieee.org/document/990517" rel="nofollow noreferrer">here</a>. All the standard image processing libraries will have it implemented. You have to use the necessary API of that library.</p>
  
  <p>This will give a bounding box around each of the detected face (it may miss out a few faces which are at different poses or illumination). You can now encode the detected face, inside the bounding box, into a vector using the features given by <a href="https://link.springer.com/article/10.3103%2FS0146411617010072" rel="nofollow noreferrer">face2vec</a>. In the face2vec model remove the classification part alone. This will give you a feature vector of size 905 dimension.</p>
  
  <p>Now given two face detections, you can find the euclidean distance between their feature vectors. If it is below a threshold say <span class="math-container">$T$</span> then, you have detected same face, else you have detected two different faces.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-11-30T07:40:02.487Z
Answers.Text: This is helpful Varsh, I will try face2vec. I am already trying encoding and graph clustering using euclidian distances. I get 128 dimension vector today. But I want to avoid Deep Learning Approaches as they are computation heavy and I want to deploy the system in a small device. 
  Is there any other possible representation of the face which can help me cluster them.
Answers.UserDisplayName: vc_dim
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-11-30T08:35:07.727Z
Answers.Text: You can use PCA to perform dimensionality reduction on raw pixels and use the reduced size vector as features. In this case you have the control over the number of dimensions thereby reducing computations for your small device.
Answers.UserDisplayName: varsh
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-11-30T06:42:08.413Z
Answers.LastActivityDate: 2018-11-30T06:42:08.413Z
Answers.UserDisplayName: varsh
Body: <p>I am having a video feed with multiple faces in it. I need to detect each face and the gender as well and assign the gender against each person. I am not sure how to uniquely identify a face as Face1 and Face2 etc. I do not need to know their names, just need to track a person in the entire video. I thought of tracking methods but people are constantly moving and changing so a box location can be occupied by another person after some frames. </p>
  
  <p>I am interested in a way where I can assign an id to a face but I am not sure how to do it. I can use Facial Recognition Based embedding on each face and track that. But that seems to be an overkill for the job. Is there any other method available or Facial Recognition/Embedding is the only method to uniquely identify people in a video?</p>
  
Tags: <deep-learning><computer-vision>
Title: Can I detect unique people in a video?
UserDisplayName: vc_dim
ThreadText: # Can I detect unique people in a video?
  
  (Asked by: username_0 on 2018-11-30)
  
  I am having a video feed with multiple faces in it. I need to detect each face and the gender as well and assign the gender against each person. I am not sure how to uniquely identify a face as Face1 and Face2 etc. I do not need to know their names, just need to track a person in the entire video. I thought of tracking methods but people are constantly moving and changing so a box location can be occupied by another person after some frames.
  
  I am interested in a way where I can assign an id to a face but I am not sure how to do it. I can use Facial Recognition Based embedding on each face and track that. But that seems to be an overkill for the job. Is there any other method available or Facial Recognition/Embedding is the only method to uniquely identify people in a video?
  
  **Tags:** deep-learning, computer-vision
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2018-11-30. Score: 2
  
  > Can I detect unique people in a video?
  
  Yes. However, for some applications commercial systems are not good enough: [German source](https://netzpolitik.org/2017/ueberwachungslabor-berlin-suedkreuz-tracking-und-gesichtserkennung-geplant/)
  
  The concepts you are looking for are:
  
  * [Optical Flow](https://en.m.wikipedia.org/wiki/Optical_flow): for two frames, define for each point how it moved
  * Object tracking: given a single object in one frame, find it in the next. Optical flow helps with that.
  * [Object detection](https://www.pyimagesearch.com/2018/11/26/instance-segmentation-with-opencv/): look at YOLO / SSD. Small introduction [here](https://medium.com/comet-app/review-of-deep-learning-algorithms-for-object-detection-c1f3d437b852).
  
  > **Comments:**
  > **username_0**: Thanks Martin, is object tracking robust if the object in interest are coming in and out of the frames. Optical flow will just track it if it is continuously in the stream.  
  > **username_1**: Correct. But Optical flow gives you a good idea where to look for new objects.  
  > **username_0**: Makes sense, Let me try that. Thanks a lot Martin.  
  
  ---
  
  ### Answer by username_2 on 2018-11-30. Score: 1
  
  You can detect faces in video frames by employing the standard method of Viola-Jones detector (though there are sophisticated ones available now). You can get the paper from [here](https://ieeexplore.ieee.org/document/990517). All the standard image processing libraries will have it implemented. You have to use the necessary API of that library.
  
  This will give a bounding box around each of the detected face (it may miss out a few faces which are at different poses or illumination). You can now encode the detected face, inside the bounding box, into a vector using the features given by [face2vec](https://link.springer.com/article/10.3103%2FS0146411617010072). In the face2vec model remove the classification part alone. This will give you a feature vector of size 905 dimension.
  
  Now given two face detections, you can find the euclidean distance between their feature vectors. If it is below a threshold say $T$ then, you have detected same face, else you have detected two different faces.
  
  > **Comments:**
  > **username_0**: This is helpful Varsh, I will try face2vec. I am already trying encoding and graph clustering using euclidian distances. I get 128 dimension vector today. But I want to avoid Deep Learning Approaches as they are computation heavy and I want to deploy the system in a small device.  Is there any other possible representation of the face which can help me cluster them.  
  > **username_2**: You can use PCA to perform dimensionality reduction on raw pixels and use the reduced size vector as features. In this case you have the control over the number of dimensions thereby reducing computations for your small device.  
  
  ---
  
  