Answers.Body: <p>Facial classification and tracking is easier compared to tracking any other motion. This is due to the fact that the face have a large number of easily identifiable features.</p>
  
  <p>Facial tracking is an additional layer on top of facial detection. Facial detection works by finding characteristics such as the cheekbones, chin, nose, eyes etc. These features are easy to detect as they have very specific properties. These points are found with the help of shadows and brightness, for example, the nose and cheekbones are highlighted, whereas the the eyes and lips have a shadows.</p>
  
  <p>Using these feature points it is possible to create a mesh to construct the face. This has been a classical way to detect and classify faces. The TrueDepth camera helps to make a better structure by providing depth with the help of an IR sensor.</p>
  
  <p><a href="https://i.sstatic.net/40R2b.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/40R2b.jpg" alt="Facial features and Facial mesh"></a></p>
  
  <p>Facial tracking makes use of this mesh to study and track changes in the facial structure. For example when you smile, it will detect the cheekbone points changing, similarly when you open your mouth, your lips will move apart.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-14T10:40:54.117Z
Answers.LastActivityDate: 2019-06-14T10:40:54.117Z
Answers.UserDisplayName: skillsmuggler
Body: <p>iPhone X allows you to look at the TrueDepth camera and reports 52 facial blendshapes like how much your eye is opened, how much your jaw is opened, etc.</p>
  
  <p>If I want to do something similar with other cameras (not TrueDepth), what are my alternative methods? Currently, I just use a simple ConvNet which takes in an image and predict 52 sigmoid values.</p>
  
  <p>What do you think could be the underlying technology behind ARKit Face Tracking?</p>
  
Tags: <deep-learning><convolutional-neural-networks><computer-vision><facial-recognition>
Title: How does ARKit's Facial Tracking work?
UserDisplayName: offchan
ThreadText: # How does ARKit's Facial Tracking work?
  
  (Asked by: username_0 on 2019-06-14)
  
  iPhone X allows you to look at the TrueDepth camera and reports 52 facial blendshapes like how much your eye is opened, how much your jaw is opened, etc.
  
  If I want to do something similar with other cameras (not TrueDepth), what are my alternative methods? Currently, I just use a simple ConvNet which takes in an image and predict 52 sigmoid values.
  
  What do you think could be the underlying technology behind ARKit Face Tracking?
  
  **Tags:** deep-learning, convolutional-neural-networks, computer-vision, facial-recognition
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-06-14. Score: 3 | Accepted answer
  
  Facial classification and tracking is easier compared to tracking any other motion. This is due to the fact that the face have a large number of easily identifiable features.
  
  Facial tracking is an additional layer on top of facial detection. Facial detection works by finding characteristics such as the cheekbones, chin, nose, eyes etc. These features are easy to detect as they have very specific properties. These points are found with the help of shadows and brightness, for example, the nose and cheekbones are highlighted, whereas the the eyes and lips have a shadows.
  
  Using these feature points it is possible to create a mesh to construct the face. This has been a classical way to detect and classify faces. The TrueDepth camera helps to make a better structure by providing depth with the help of an IR sensor.
  
  [![Facial features and Facial mesh](https://i.sstatic.net/40R2b.jpg)](https://i.sstatic.net/40R2b.jpg)
  
  Facial tracking makes use of this mesh to study and track changes in the facial structure. For example when you smile, it will detect the cheekbone points changing, similarly when you open your mouth, your lips will move apart.
  
  
  ---
  
  