Answers.Body: <p>I also recommend you take a look at the following work by Uber AI Labs who used an interesting approach to computer games:</p>
  
  <p><a href="https://eng.uber.com/deep-neuroevolution/" rel="nofollow noreferrer">https://eng.uber.com/deep-neuroevolution/</a></p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-02-18T21:28:14.100Z
Answers.Text: Nice link.  This topic has definitely come up on the forum.
Answers.UserDisplayName: DukeZhou
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2018-02-18T02:19:20.447Z
Answers.LastActivityDate: 2018-02-18T02:19:20.447Z
Answers.UserDisplayName: Mike AI
Answers.Body: <p>I recommend you read up on reinforcement learning. Seeing how AirHockey is similar to the old Atari game Pong, here is a write-up (with code) about how to implement a simple neural network, that plays the game; <a href="http://karpathy.github.io/2016/05/31/rl/" rel="nofollow noreferrer">Deep Reinforcement Learning: Pong from Pixels</a>.</p>
  
  <p>As far as the choice of layers go, again, you should read up on deep reinforcement learning, perhaps study some existing models that work, for a similar task and take their architecture as your starting point. The above link is a nice introduction into such thinking.</p>
  
  <p>As a continuation, I would recomend you employ one of the well known, and developed, libraries, such as <a href="https://www.tensorflow.org" rel="nofollow noreferrer">TensorFlow</a> or <a href="http://pytorch.org" rel="nofollow noreferrer">pyTorch</a>. They will allow you to run your models on GPU, like you desire.</p>
  
  <p>For a toolkit, that specializes to reinforcement learning, see <a href="https://github.com/openai/gym" rel="nofollow noreferrer">openAI gym</a>.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-09-29T17:07:59.930Z
Answers.Text: The first link was immensely helpful, and based on the information, I believe I'll go with the convolutional neural network like my professor recommended, using the preprocessed image frames as input. However, I believe based on my updated understanding that my 2 outputs should be the velocity of moving up and to the right, (after subtracting 0.5 from each and normalizing to the highest velocity our robot can support), that way it's _action_ based rather than location based.
Answers.UserDisplayName: Patrick Roberts
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-09-29T17:13:06.497Z
Answers.Text: I'm glad. Hope your project goes well.
Answers.UserDisplayName: Žiga Sajovic
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-09-29T09:41:59.020Z
Answers.LastActivityDate: 2017-09-29T09:41:59.020Z
Answers.UserDisplayName: Žiga Sajovic
Body: <p>For implementing a neural network algorithm that can play air hockey, I had two ideas for input, and I'm trying to figure out which design would be most viable.</p>
  
  <p>The output must be two analog values that dictate the best position on half of the table for the robot to be at a specific point in time, evaluated 60 times per second.</p>
  
  <p>Having consulted with a professor who has experience with implementing parallel algorithms, it was recommended to me to use a convolutional neural network with a single hidden layer, and directly process the image data as the input layer, after processing it to visualize a direct view of the table and somehow emphasize the puck and mallets with pre-processing. I have already started work on this and have successfully implemented object detection for the puck and mallets using OpenCV to get the center coordinates for all 3 entities.</p>
  
  <p>However, having been able to successfully and accurately pre-process these data at 60 times per second, my thought was to feed those (after normalizing the values using the error function) directly on the input layer and possibly implement a deep learning algorithm that employs more than one hidden layer. The problem is that I don't have any experience implementing neural networks, so I'm not even sure what type of layer would be best for this, or how I should seed the weights.</p>
  
  <p>Another reason I want to consider my idea is that given the relatively few inputs and outputs, probably won't need a GPU to execute forward-propagation 60 times per second, whereas with the convolutional network my professor recommended, I <strong>know</strong> I'll need to implement using CUDA somehow.</p>
  
  <p>Which of the two input methods would be most recommended for this, and if I were to try and implement my idea, what types of layers should I consider? Also any recommendations for existing frameworks to use for either approach would be highly appreciated.</p>
  
Tags: <neural-networks><ai-design><game-ai><hidden-layers><real-time>
Title: What type of neural network would be most feasible for playing a realtime game?
UserDisplayName: Patrick Roberts
ThreadText: # What type of neural network would be most feasible for playing a realtime game?
  
  (Asked by: username_0 on 2017-09-28)
  
  For implementing a neural network algorithm that can play air hockey, I had two ideas for input, and I'm trying to figure out which design would be most viable.
  
  The output must be two analog values that dictate the best position on half of the table for the robot to be at a specific point in time, evaluated 60 times per second.
  
  Having consulted with a professor who has experience with implementing parallel algorithms, it was recommended to me to use a convolutional neural network with a single hidden layer, and directly process the image data as the input layer, after processing it to visualize a direct view of the table and somehow emphasize the puck and mallets with pre-processing. I have already started work on this and have successfully implemented object detection for the puck and mallets using OpenCV to get the center coordinates for all 3 entities.
  
  However, having been able to successfully and accurately pre-process these data at 60 times per second, my thought was to feed those (after normalizing the values using the error function) directly on the input layer and possibly implement a deep learning algorithm that employs more than one hidden layer. The problem is that I don't have any experience implementing neural networks, so I'm not even sure what type of layer would be best for this, or how I should seed the weights.
  
  Another reason I want to consider my idea is that given the relatively few inputs and outputs, probably won't need a GPU to execute forward-propagation 60 times per second, whereas with the convolutional network my professor recommended, I **know** I'll need to implement using CUDA somehow.
  
  Which of the two input methods would be most recommended for this, and if I were to try and implement my idea, what types of layers should I consider? Also any recommendations for existing frameworks to use for either approach would be highly appreciated.
  
  **Tags:** neural-networks, ai-design, game-ai, hidden-layers, real-time
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_3 on 2017-09-29. Score: 2 | Accepted answer
  
  I recommend you read up on reinforcement learning. Seeing how AirHockey is similar to the old Atari game Pong, here is a write-up (with code) about how to implement a simple neural network, that plays the game; [Deep Reinforcement Learning: Pong from Pixels](http://karpathy.github.io/2016/05/31/rl/).
  
  As far as the choice of layers go, again, you should read up on deep reinforcement learning, perhaps study some existing models that work, for a similar task and take their architecture as your starting point. The above link is a nice introduction into such thinking.
  
  As a continuation, I would recomend you employ one of the well known, and developed, libraries, such as [TensorFlow](https://www.tensorflow.org) or [pyTorch](http://pytorch.org). They will allow you to run your models on GPU, like you desire.
  
  For a toolkit, that specializes to reinforcement learning, see [openAI gym](https://github.com/openai/gym).
  
  > **Comments:**
  > **username_0**: The first link was immensely helpful, and based on the information, I believe I'll go with the convolutional neural network like my professor recommended, using the preprocessed image frames as input. However, I believe based on my updated understanding that my 2 outputs should be the velocity of moving up and to the right, (after subtracting 0.5 from each and normalizing to the highest velocity our robot can support), that way it's _action_ based rather than location based.  
  > **username_3**: I'm glad. Hope your project goes well.  
  
  ---
  
  ### Answer by username_1 on 2018-02-18. Score: 1
  
  I also recommend you take a look at the following work by Uber AI Labs who used an interesting approach to computer games:
  
  <https://eng.uber.com/deep-neuroevolution/>
  
  > **Comments:**
  > **username_2**: Nice link.  This topic has definitely come up on the forum.  
  
  ---
  
  