Answers.Body: <p>The common options are:</p>
  
  <ol>
  <li>Implement a simple simulation of the game, perhaps without graphics. Train the agent on the simulation. This is usually not very satisfying, because the agent won't play the <em>real</em> game.</li>
  <li>Use an emulator that allows you to inspect the game's memory, and input values directly at the controls. <a href="http://tasvideos.org/BizHawk.html" rel="nofollow noreferrer">BizHawk</a> is a good choice for SNES and I think also for N64 emulation. Use a <a href="https://www.geeksforgeeks.org/pipe-system-call/" rel="nofollow noreferrer">pipe</a> to connect this program with your agent. Pass across the pipe any key values from the game's memory that represent the state of the game, and send back actions that represent the inputs the agent makes in that state. Usually this is the best option.</li>
  <li>Scrape the screen in real time. Tools like <a href="https://github.com/muldjord/skyscraper" rel="nofollow noreferrer">SkyScraper</a> will allow you to capture the raw pixels displayed in a window. You should then be able to pass these pixels to your agents via a pipe. You may be able to use tools like <a href="https://www.seleniumhq.org/" rel="nofollow noreferrer">Selenium</a> to pass values back from the agent to the game. This route is usually a lot more intensive than option 2, but is the most realistic. It may also require very good hardware to keep up with the game in real time.</li>
  </ol>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-10-08T00:28:24.460Z
Answers.LastActivityDate: 2019-10-08T00:28:24.460Z
Answers.UserDisplayName: John Doucette
Body: <p>I would like to work on a project where I teach an NN to play N64 games. 
  To my current understanding, I would need an emulator?</p>
  
  <p>I can do the Machine Learning side of it, im just unsure how I can give the NN access to the game's controls, such as left, right, up or down?</p>
  
  <p>Where could I find more information on doing so and is using an Emulator the right path to take?</p>
  
Tags: <neural-networks><reinforcement-learning>
Title: How can I teach a computer to play N64 games using Neural Nets?
UserDisplayName: Eoin Mc Allister
ThreadText: # How can I teach a computer to play N64 games using Neural Nets?
  
  (Asked by: username_0 on 2019-10-07)
  
  I would like to work on a project where I teach an NN to play N64 games.
  To my current understanding, I would need an emulator?
  
  I can do the Machine Learning side of it, im just unsure how I can give the NN access to the game's controls, such as left, right, up or down?
  
  Where could I find more information on doing so and is using an Emulator the right path to take?
  
  **Tags:** neural-networks, reinforcement-learning
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-10-08. Score: 3 | Accepted answer
  
  The common options are:
  
  1. Implement a simple simulation of the game, perhaps without graphics. Train the agent on the simulation. This is usually not very satisfying, because the agent won't play the *real* game.
  2. Use an emulator that allows you to inspect the game's memory, and input values directly at the controls. [BizHawk](http://tasvideos.org/BizHawk.html) is a good choice for SNES and I think also for N64 emulation. Use a [pipe](https://www.geeksforgeeks.org/pipe-system-call/) to connect this program with your agent. Pass across the pipe any key values from the game's memory that represent the state of the game, and send back actions that represent the inputs the agent makes in that state. Usually this is the best option.
  3. Scrape the screen in real time. Tools like [SkyScraper](https://github.com/muldjord/skyscraper) will allow you to capture the raw pixels displayed in a window. You should then be able to pass these pixels to your agents via a pipe. You may be able to use tools like [Selenium](https://www.seleniumhq.org/) to pass values back from the agent to the game. This route is usually a lot more intensive than option 2, but is the most realistic. It may also require very good hardware to keep up with the game in real time.
  
  
  ---
  
  