Body: <p>So, I’m looking into some dynamic ways in which one can drive the behavior of a video game character. Specifically an NPC (Non playable character) that will be observable from the players point of view. Something I’d like to clarify from the start is what I mean by behavior. Since video games are understood visually, then I would qualify behavior to be anything visual, such as gestures, mannerisms or actions in any local space. </p>
  
  <p>Let’s take a common archetype as an example. We’ll say we want the behavior of a villain. My first thought, was to use videos as training data. Videos of specific subjects or actors in a villainous scene (think Frankenstein, Dracula, Emperor Palpatine in Star Wars etc...) in hopes that an understanding of their mannerisms, body language and gestures could be captured and later applied to 3D animation dynamically.</p>
  
  <p>I do understand that anything 3D typically requires rigging and animation. I’m currently not exactly sure how to marshal the data from one format (video analyses) to 3D animation. I thought I’d start working on the concept from high level first.</p>
  
  <p>Any thoughts?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-11-05T21:02:36.470Z
Comments.Text: Is it safe to assume that this is just about animation, and not action choices? So you might be interested in things like motion style transfer e.g. https://www.youtube.com/watch?v=AvVQZjygQBQ - it's not the only thing that might be of interest in this topic area.
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-11-06T14:24:00.477Z
Comments.Text: Right, but for those of us who do not want to use a traditional motion capture method, couldn't one use semantic segmentation mapping for video footage to create the needed labels for input into an algorithm? A google search of "Video Semantic Segmentation" reveals how this can be done. Its just, I don't know which algorithm to use for capturing the expressiveness of the motion. Can we approach this in a non-traditional way?
Comments.UserDisplayName: junfanbl
Tags: <neural-networks>
Title: Can a Video Game Characters Behavior be directed by a NN?
UserDisplayName: junfanbl
ThreadText: # Can a Video Game Characters Behavior be directed by a NN?
  
  (Asked by: username_0 on 2019-11-05)
  
  So, I’m looking into some dynamic ways in which one can drive the behavior of a video game character. Specifically an NPC (Non playable character) that will be observable from the players point of view. Something I’d like to clarify from the start is what I mean by behavior. Since video games are understood visually, then I would qualify behavior to be anything visual, such as gestures, mannerisms or actions in any local space.
  
  Let’s take a common archetype as an example. We’ll say we want the behavior of a villain. My first thought, was to use videos as training data. Videos of specific subjects or actors in a villainous scene (think Frankenstein, Dracula, Emperor Palpatine in Star Wars etc...) in hopes that an understanding of their mannerisms, body language and gestures could be captured and later applied to 3D animation dynamically.
  
  I do understand that anything 3D typically requires rigging and animation. I’m currently not exactly sure how to marshal the data from one format (video analyses) to 3D animation. I thought I’d start working on the concept from high level first.
  
  Any thoughts?
  
  **Tags:** neural-networks
  > **Comments:**
  > **username_1**: Is it safe to assume that this is just about animation, and not action choices? So you might be interested in things like motion style transfer e.g. https://www.youtube.com/watch?v=AvVQZjygQBQ - it's not the only thing that might be of interest in this topic area.  
  > **username_0**: Right, but for those of us who do not want to use a traditional motion capture method, couldn't one use semantic segmentation mapping for video footage to create the needed labels for input into an algorithm? A google search of "Video Semantic Segmentation" reveals how this can be done. Its just, I don't know which algorithm to use for capturing the expressiveness of the motion. Can we approach this in a non-traditional way?  
  
  ---
  
  ## 0 answers
  
  