Answers.Body: <p>I recommend preprocessing images and feeding pixel values of several combined images. Some ideas:</p>
  
  <ol>
  <li><p>Preprocess all images to grayscale if possible. It’ll reduce the number of input neurons. (As long as this step doesn’t introduce large overhead)</p></li>
  <li><p>Select some <span class="math-container">$\gamma$</span> value such that 0 &lt; <span class="math-container">$\gamma$</span> &lt; 1. Generate (ie. Select from your game) <span class="math-container">$n$</span> sequential images. For the <span class="math-container">$k$</span>th image in the sequence, multiply every pixel value by <span class="math-container">$\gamma^{n-k-1}$</span>. This assumes we index <span class="math-container">$k$</span> starting at zero. </p></li>
  <li><p>Sum the pixel values of all processed images with clip ~ [0, 255] (for a valid range of values)</p></li>
  </ol>
  
  <p>This will yield a single image where stationary pixels will be summed to create brighter / more saturated spots, where moving objects will have “shadows” or “tails” which are faded with each time step (<span class="math-container">$\gamma$</span> is the “fading factor” so to speak). </p>
  
  <p><em>Image input:</em> As long as all values are on a comparable scale, it shouldn’t really matter whether inputs are on range [-1, 1] or [0, 1], but since you’ll be using pixel values, they will all be positive. So normalizing the pixel values will yield a range [0, 1].  </p>
  
  <p><strong>Note:</strong> this kind of processing can probably be done iteratively with greater efficiency by summing, then multiplying by gamma at each time step. Then you can implement it online.</p>
  
  <p>Now consider what you want the OUTPUT of the network to be. If you want the agent to take an action after processing the inputs, your output later should consist of one neuron per discrete action (ie, each “button” that can be pushed). I will limit my answer to discrete actions since that is likely the most useful answer for this question. </p>
  
  <p>Finally, you asked about if the network can “remember things,” like “holding down a key.”  This question is a bit vague, but let me try to answer. It sounds like you were considering using this as an INPUT to the network. In theory, you could use a similar implementation (ie. At every time step measure if the button was pressed. Perhaps use 1 if pressed 0 otherwise. Decay at every time step and sum. With n time steps, the sum will have a max value of <span class="math-container">$\sum_{n}(\gamma^{n-1})$</span>). Remember to decay by <span class="math-container">$n-k-1$</span>, with <span class="math-container">$k$</span> starting at zero. You  don’t have to actually decay this value, but decaying by a factor of gamma helps the network know if for instance the button was pressed near the first frame or the last frame. </p>
  
  <p>With that said, I don’t know if you want to use this as an input. If the AI is meant to have more information than the opponent, than I suppose you can. But then the agent will not be learning from the same information as the opponent. Also, if holding the button produces a clearly visible effect, that information would be captured in the images already, so might be a redundant input. </p>
  
  <p>These ideas are not the only implementation, but can get you going. It sounds like you’ll need a measure of reward and likely need to structure this as an RL problem. The details of that are beyond the scope of this post and I don’t want to get too afield of your original question. Just note that comparing to the players output may not give you the results you want, and even if it did, your network will be limited to learning only to mimic the other player. Using a measure of reward will allow your agent to theoretically advance beyond the skill of its opponent by taking actions that maximize reward, even if the opponent would not have thought to take that action. </p>
  
  <p>I hope this helps. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-28T11:24:49.597Z
Answers.Text: Thank you for your answer, you definitely helped a lot! This seems to be very efficient and elegant. I did not understand all of it, though. First of all, in your step 2, do you mean gamma^(n-k-1) or (gamma^n)-k-1 ? \n and does step 3 mean that I use k images just to calculate one set of inputs ? or do I use k times the amount of pixels as inputs and feed in the last k images ?
Answers.UserDisplayName: Cr3ative
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-28T12:12:23.957Z
Answers.Text: @Cr3ative I updated my answer with Mathjax for clarity. Step 3 means use k images to calculate one set of inputs. You will sum the values element wise for all images. Ideally, we want to keep the number of inputs down to a reasonable number. Using k images and combining them through an exponential decay gives a single image of the same size that gives a sort of glimpse into the past.
Answers.UserDisplayName: Hanzy
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-05-04T14:28:00.840Z
Answers.Text: The glimpse into the past thing is very cool and promising. My goal is actually to mimic the player behaviour and not surpass him in terms of skill. - at least as a first step. I want to see it behave in a way you can recognize from the players behaviour. I will look into RL. If I‘ll be able to realize this you‘ll get the link first ;)
Answers.UserDisplayName: Cr3ative
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-05-05T01:54:51.973Z
Answers.Text: @Cr3ative for what it’s worth, some people just stack $n$ images depth wise (ie, each image is another channel, all being fed into a CNN). I can’t attest to the relative efficacy of the two methods, but simply stacking them along the channel dimension seems pretty common. You’ll see that employed throughout this dissertation: https://deepblue.lib.umich.edu/bitstream/handle/2027.42/136931/guoxiao_1.pdf?sequence=1&isAllowed=y
Answers.UserDisplayName: Hanzy
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-28T03:12:19.397Z
Answers.LastActivityDate: 2019-04-28T15:22:34.793Z
Answers.LastEditDate: 2019-04-28T15:22:34.793Z
Answers.UserDisplayName: Hanzy
Answers.Body: <p>The most generic approach is to input all pixels as you have suggested. A CNN would be the best architecture for that. To provide information like speed or velocity, you can feed more than one frame to the CNN (e.g. the last 5 frames or whatever provides enough information). The CNN can learn movement information by comparing those images.</p>
  
  <p>If you want to store additional information (like an inventory item), an input neuron for each value would be an option. You can also look up LSTM (long-short term memory) models, but for your specific situation a hardcoded neuron would be the easier solution.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-28T07:18:27.547Z
Answers.Text: Thank you for your answer. Using more than one frame to detect movement sounds very elegant compared to inputting some calculated speed value. I will definitely look into the field of CNNs
Answers.UserDisplayName: Cr3ative
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-04-27T19:11:02.067Z
Answers.LastActivityDate: 2019-04-27T19:11:02.067Z
Answers.UserDisplayName: Demento
Body: <p>I want to create a simple game which basically consists of 2d circles shooting smaller circles at each other (to make hitbox detection easier for the start). My goal is to create an ai which adapts its own behaviour to the player‘s. For that, i want to use a NN as brain. Every frame, the NN is fed with the same inputs as the player and his output is compared to the players output. (outputs in this case are pressed keys like the up-arrow) 
  As inputs, I want to use a couple different important factors:
  for example, the direction of the enemy player as number from 0 to 1</p>
  
  <p>I also want to input the direction, size and speed of enemy’s and own projectiles and this is where my problem lies. If there was only one bullet per player, it would be easy but I want the number of bullets to be variable so the number of input neurons would have to be variable.</p>
  
  <p>My approaches:
  1) use a big amount of neurons and set unused ones to 0
  ( not elegant at all)
  2) Instead of specific values, just use all the pixels‘ rgb values as inputs (would limit the game as colours would deliver all the information) (+factors like speed and direction would probably not have any impact)</p>
  
  <p>Is there a more promising approach to this problem ?
  I hope you can give me some inspiration.</p>
  
  <p>Also, is there a difference in ranging input values between 0/1 or -1/1 ?</p>
  
  <p>Thank you in advance, Mo</p>
  
  <p>Edit: In case there aren‘t enough questions for you, is there a way to make the NN remember things ? For example, if I added a mechanic to the game which involves holding a key, I would add an input neuron which inputs 1 if the certain key is pressed and 0 if it isn‘t but I doubt that would work.</p>
  
Tags: <neural-networks><game-ai><neurons><java>
Title: Neural Network with varying inputs (for a game ai)
UserDisplayName: Cr3ative
ThreadText: # Neural Network with varying inputs (for a game ai)
  
  (Asked by: username_0 on 2019-04-26)
  
  I want to create a simple game which basically consists of 2d circles shooting smaller circles at each other (to make hitbox detection easier for the start). My goal is to create an ai which adapts its own behaviour to the player‘s. For that, i want to use a NN as brain. Every frame, the NN is fed with the same inputs as the player and his output is compared to the players output. (outputs in this case are pressed keys like the up-arrow)
  As inputs, I want to use a couple different important factors:
  for example, the direction of the enemy player as number from 0 to 1
  
  I also want to input the direction, size and speed of enemy’s and own projectiles and this is where my problem lies. If there was only one bullet per player, it would be easy but I want the number of bullets to be variable so the number of input neurons would have to be variable.
  
  My approaches:
  1) use a big amount of neurons and set unused ones to 0
  ( not elegant at all)
  2) Instead of specific values, just use all the pixels‘ rgb values as inputs (would limit the game as colours would deliver all the information) (+factors like speed and direction would probably not have any impact)
  
  Is there a more promising approach to this problem ?
  I hope you can give me some inspiration.
  
  Also, is there a difference in ranging input values between 0/1 or -1/1 ?
  
  Thank you in advance, Mo
  
  Edit: In case there aren‘t enough questions for you, is there a way to make the NN remember things ? For example, if I added a mechanic to the game which involves holding a key, I would add an input neuron which inputs 1 if the certain key is pressed and 0 if it isn‘t but I doubt that would work.
  
  **Tags:** neural-networks, game-ai, neurons, java
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2019-04-28. Score: 2 | Accepted answer
  
  I recommend preprocessing images and feeding pixel values of several combined images. Some ideas:
  
  1. Preprocess all images to grayscale if possible. It’ll reduce the number of input neurons. (As long as this step doesn’t introduce large overhead)
  2. Select some $\gamma$ value such that 0 < $\gamma$ < 1. Generate (ie. Select from your game) $n$ sequential images. For the $k$th image in the sequence, multiply every pixel value by $\gamma^{n-k-1}$. This assumes we index $k$ starting at zero.
  3. Sum the pixel values of all processed images with clip ~ [0, 255] (for a valid range of values)
  
  This will yield a single image where stationary pixels will be summed to create brighter / more saturated spots, where moving objects will have “shadows” or “tails” which are faded with each time step ($\gamma$ is the “fading factor” so to speak).
  
  *Image input:* As long as all values are on a comparable scale, it shouldn’t really matter whether inputs are on range [-1, 1] or [0, 1], but since you’ll be using pixel values, they will all be positive. So normalizing the pixel values will yield a range [0, 1].
  
  **Note:** this kind of processing can probably be done iteratively with greater efficiency by summing, then multiplying by gamma at each time step. Then you can implement it online.
  
  Now consider what you want the OUTPUT of the network to be. If you want the agent to take an action after processing the inputs, your output later should consist of one neuron per discrete action (ie, each “button” that can be pushed). I will limit my answer to discrete actions since that is likely the most useful answer for this question.
  
  Finally, you asked about if the network can “remember things,” like “holding down a key.” This question is a bit vague, but let me try to answer. It sounds like you were considering using this as an INPUT to the network. In theory, you could use a similar implementation (ie. At every time step measure if the button was pressed. Perhaps use 1 if pressed 0 otherwise. Decay at every time step and sum. With n time steps, the sum will have a max value of $\sum\_{n}(\gamma^{n-1})$). Remember to decay by $n-k-1$, with $k$ starting at zero. You don’t have to actually decay this value, but decaying by a factor of gamma helps the network know if for instance the button was pressed near the first frame or the last frame.
  
  With that said, I don’t know if you want to use this as an input. If the AI is meant to have more information than the opponent, than I suppose you can. But then the agent will not be learning from the same information as the opponent. Also, if holding the button produces a clearly visible effect, that information would be captured in the images already, so might be a redundant input.
  
  These ideas are not the only implementation, but can get you going. It sounds like you’ll need a measure of reward and likely need to structure this as an RL problem. The details of that are beyond the scope of this post and I don’t want to get too afield of your original question. Just note that comparing to the players output may not give you the results you want, and even if it did, your network will be limited to learning only to mimic the other player. Using a measure of reward will allow your agent to theoretically advance beyond the skill of its opponent by taking actions that maximize reward, even if the opponent would not have thought to take that action.
  
  I hope this helps.
  
  > **Comments:**
  > **username_0**: Thank you for your answer, you definitely helped a lot! This seems to be very efficient and elegant. I did not understand all of it, though. First of all, in your step 2, do you mean gamma^(n-k-1) or (gamma^n)-k-1 ? \n and does step 3 mean that I use k images just to calculate one set of inputs ? or do I use k times the amount of pixels as inputs and feed in the last k images ?  
  > **username_1**: @username_0 I updated my answer with Mathjax for clarity. Step 3 means use k images to calculate one set of inputs. You will sum the values element wise for all images. Ideally, we want to keep the number of inputs down to a reasonable number. Using k images and combining them through an exponential decay gives a single image of the same size that gives a sort of glimpse into the past.  
  > **username_0**: The glimpse into the past thing is very cool and promising. My goal is actually to mimic the player behaviour and not surpass him in terms of skill. - at least as a first step. I want to see it behave in a way you can recognize from the players behaviour. I will look into RL. If I‘ll be able to realize this you‘ll get the link first ;)  
  > **username_1**: @username_0 for what it’s worth, some people just stack $n$ images depth wise (ie, each image is another channel, all being fed into a CNN). I can’t attest to the relative efficacy of the two methods, but simply stacking them along the channel dimension seems pretty common. You’ll see that employed throughout this dissertation: https://deepblue.lib.umich.edu/bitstream/handle/2027.42/136931/guoxiao_1.pdf?sequence=1&isAllowed=y  
  
  ---
  
  ### Answer by username_2 on 2019-04-27. Score: 2
  
  The most generic approach is to input all pixels as you have suggested. A CNN would be the best architecture for that. To provide information like speed or velocity, you can feed more than one frame to the CNN (e.g. the last 5 frames or whatever provides enough information). The CNN can learn movement information by comparing those images.
  
  If you want to store additional information (like an inventory item), an input neuron for each value would be an option. You can also look up LSTM (long-short term memory) models, but for your specific situation a hardcoded neuron would be the easier solution.
  
  > **Comments:**
  > **username_0**: Thank you for your answer. Using more than one frame to detect movement sounds very elegant compared to inputting some calculated speed value. I will definitely look into the field of CNNs  
  
  ---
  
  