Disney Combines CGI with Neural Rendering to Tackle the ‘Uncanny Valley’
Can someone tell me why almost all of the “realistic” 3D rendering is about superficial skin deformation?<p>Hear me out:<p>- Train an AI to determine fascial muscle activation from mocap data<p>- Build your animation pipeline to use muscle activation to drive model deformation<p>- Take the actor’s mocap acting and have the system convert their muscle movements to the 3d model’s muscle movements<p>- Tackle the uncanny valley by refining how well we implement the bone&#x2F;skin&#x2F;muscle structure of human faces<p>- Bonus points: Voice. So much of the sound of a person’s voice is about the shaped made by their mouth and throat (and torso). We <i>could</i> have an AI recreate the whole vocal path for a 3D character, resulting in more realistic voice acting.