A virtual reality (VR) or augmented reality (AR) head mounted display (HMD) includes various facial sensors, such as cameras, that capture images of portions of the user's face outside of the HMD. For example, multiple facial sensors capture images of a portion of the user's face below the HMD. Through image analysis, points of the portion of the user's face are identified from the images and their movement is tracked. The identified points are mapped to a three dimensional model of a face. Additionally, a parametric representation of the user's face is determined for each captured image, resulting in various representations indicating the user's facial expressions. From the parametric representations and transforms mapping the captured images to three dimensions, a rendering model is used and applied to the three dimensional model of the face to render the user's facial expressions.
Generating a graphical representation of a face of a user wearing a head mounted display