Answers.Body: <p>It depends on the game environment and on the model being trained.</p>
  <ul>
  <li>If you are training an agent that uses vision to decide action, then typically you need a copy of the rendered screen:
  <ul>
  <li>If that comes from a modern game using GPU acceleration to generate output, then you will likely need to leave that as is, and also transfer the rendered output back to the main memory, so it will likely be slower than running the game normally. There may be game settings that help improve speed.</li>
  <li>If the output is from a game emulator for older games (such as Atari), then it probably won't use a GPU, and you will not need to spend time rendering to the screen. The emulator may be able to run in a mode that doesn't render, whilst still giving in-memory access to the screen buffer. It may also be able to run faster than real time. It will depend on emulator configuration settings that are available to you.</li>
  </ul>
  </li>
  <li>If the game is written &quot;natively&quot; in a simulator or semi-native environment like Pygame or Box2D, you should be able to disable rendering and save the time. This is how a lot of OpenAI Gym's environments are set up, and these do run a lot faster when not rendering - a simple flag you can set when starting the environment and/or starting an episode. You may even notice with these environments that putting the view window for the environment behind another window will speed things up considerably (but for automation it is more reliable to disable rendering).</li>
  </ul>
  <p>When the environments have been set up via a shared library, like Open AI's Gym, then the internal logic for deciding rendering (or not) will be different for each environment, but there may be a standardised config/method argument to determine whether the game engine is rendered to screen. That could apply to nearly all the environments available in the library. You should check the documentation if for example you are trying out NEAT on some Atari games that have been ready packaged for use with computer agents.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-05T05:18:08.390Z
Answers.Text: Thanks for the detailed explanation! Talking about OpenAI, when they created openAI 5 for the "Dota 2" game, did they create a copy of it without rendering to allow fast training?
Answers.UserDisplayName: snatchysquid
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-05T07:29:13.850Z
Answers.Text: @snatchysquid: I don't know, but my understanding is that the RL environment for DoTA is not based on the pixels, but on the object models in the game via an API, which implies the kind of low-level access where that is possible.
Answers.UserDisplayName: Neil Slater
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-04T18:04:28.970Z
Answers.LastActivityDate: 2022-06-06T07:59:17.603Z
Answers.LastEditDate: 2022-06-06T07:59:17.603Z
Answers.UserDisplayName: Neil Slater
Answers.Body: <h2>Bypassing graphics</h2>
  <p>As mentioned in Neil Slater's answer, manipulating the engine to bypass graphics rendering to speed up AI simulation can be a valid approach. I have done that myself.</p>
  <p>But there are some caveats to consider/work around:</p>
  <ul>
  <li>The graphics subsystem can be used for core gameplay programming as well. For example, the picking problem (Which object in the scene is the user selecting/clicking on?) <a href="http://www.opengl-tutorial.org/miscellaneous/clicking-on-objects/picking-with-an-opengl-hack/" rel="nofollow noreferrer">can be solved by rendering and evaluating framebuffers</a>.</li>
  <li>If the game has an FPS limiter, then you need to raise that FPS limit. Otherwise it quickly becomes the new bottleneck.</li>
  <li>If the game also has real-time timers (i.e. wait five seconds to trigger some event), then your ultra-fast gameplay simulation can cause unintended effects/bugs. This is especially true if you break the aforementioned FPS limit.</li>
  <li>Sometimes games can experience bugs and crashes when running at high framerates. (<a href="https://www.pcgamingwiki.com/wiki/Peter_Jackson%27s_King_Kong_Gamer%27s_Edition#Game_breaking_bugs_at_high_framerates" rel="nofollow noreferrer">Take this PCGamingWiki entry on
  <em>Peter Jackson's King Kong Gamer's Edition</em> as an example.</a>) With high framerates, gameplay simulation may operate with very small timesteps and glitches in physics and collision detection become more likely. Such effects will only intensify with ultra-high framerates from the accelerated AI simulation.</li>
  </ul>
  <h3>DLL injection</h3>
  <p>It is not uncommon for game mods to <a href="https://www.upguard.com/blog/dll-hijacking" rel="nofollow noreferrer">hijack or replace DLLs</a>, including those of the graphics API. If you cannot manipulate the engine itself, then perhaps you can inject an alternative graphics API implementation that consists of stubs and mockups.</p>
  <h2>Bypass whatever is irrelevant to gameplay</h2>
  <p>Depending on the game and the goals of your project, you might take even more drastic steps to accelerate the simulation:</p>
  <ul>
  <li>Bypass the audio layer as well. Besides graphics, audio is another major output channel that is often irrelevant to AI simulation.</li>
  <li>Skip through cutscenes and dialogue, like a speedrunner would do.</li>
  <li>If walking is not essential to gameplay, let the avatar teleport instead of simulating all the walking.</li>
  <li>Skip animations in general. Always try to fast-forward to the &quot;gameplay outcome&quot; of an action.</li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-06T12:03:40.023Z
Answers.LastActivityDate: 2022-06-06T12:03:40.023Z
Answers.UserDisplayName: meekee7
Body: <p>Youtube was recently suggesting to me videos of people training NEAT neural networks for video games. I've noticed that often the training process was quite slow (for example in <a href="https://www.youtube.com/watch?v=a8Bo2DHrrow" rel="nofollow noreferrer">this</a> Trackmania example).</p>
  <p>Is there a way (algorithmic approach or an idea) to easily simulate video games, without actually rendering the pixels on screen and making the training much quicker? In addition to that, if you also know of a tool that does that, please, share it with me.</p>
  <hr />
  <p><strong>Update:</strong></p>
  <p>Following the Trackmania example, I found out the youtuber uses a standard tool called &quot;<a href="https://donadigo.com/tminterface/" rel="nofollow noreferrer">TMInterface</a>&quot;, and the webpage of the tool states it is a &quot;TAS tool&quot; - <a href="https://en.wikipedia.org/wiki/Tool-assisted_speedrun#Method" rel="nofollow noreferrer">Tool-assisted speedrun</a>.</p>
  <p>I'll investigate how such tools work and whats the idea behind them (and if indeed they do what I think they do). Will update!</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-06-05T09:21:04.883Z
Comments.Text: If you're asking for a tool, this question is off-topic. If you're looking for an approach/algorithm, then it could be on-topic. Questions that ask for software libraries should be asked on https://softwarerecs.stackexchange.com/ or DS SE. We focus on the theoretical/conceptual, philosophical and social aspects of AI (although we have had very few questions on the latter 2 topics recently). See https://ai.stackexchange.com/help/on-topic. Please, edit your post to clarify that you're not asking for a tool (unless that's an additional request), if you don't want your post to be closed.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-06-06T16:45:55.303Z
Comments.Text: I don't think this is a tool as much as "why the process" sort of question.  The deepmind atari, afaict, didn't use the screen, but used a representation in memory.  Some versions (starcraft) can also look at other variables in memory like positions of objects.
Comments.UserDisplayName: EngrStudent
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-06-06T17:02:39.670Z
Comments.Text: @EngrStudent How can you access the variables in memory and interact with the game without the game wasting time rendering?
Comments.UserDisplayName: snatchysquid
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-06-06T18:13:42.760Z
Comments.Text: I think some of the answers have them. The screen is a pixel map. Rendering seeks to fill in all the pixels with intensity values. The internal representation that the computer holds would be very inefficient if it was all pixels. The JPEG image is discrete cosine coefficients. If you can expose the learner to the input to the renderer instead of its output, then you can give it a compressed form before that form is burdening the silicon.
Comments.UserDisplayName: EngrStudent
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-06-07T09:42:48.953Z
Comments.Text: Note that graphics don't have to be rendered in real time and don't have to be rendered to the screen! You may be able to patch the game to render to a framebuffer and run as fast as possible (instead of slowing itself down to 100% normal speed)
Comments.UserDisplayName: Criticize SE actions means ban
Tags: <reinforcement-learning><machine-learning><neat><resource-request>
Title: Is there a way to easily simulate video games, without actually rendering the pixels on screen?
UserDisplayName: snatchysquid
ThreadText: # Is there a way to easily simulate video games, without actually rendering the pixels on screen?
  
  (Asked by: username_0 on 2022-06-04)
  
  Youtube was recently suggesting to me videos of people training NEAT neural networks for video games. I've noticed that often the training process was quite slow (for example in [this](https://www.youtube.com/watch?v=a8Bo2DHrrow) Trackmania example).
  
  Is there a way (algorithmic approach or an idea) to easily simulate video games, without actually rendering the pixels on screen and making the training much quicker? In addition to that, if you also know of a tool that does that, please, share it with me.
  
  ---
  
  **Update:**
  
  Following the Trackmania example, I found out the youtuber uses a standard tool called "[TMInterface](https://donadigo.com/tminterface/)", and the webpage of the tool states it is a "TAS tool" - [Tool-assisted speedrun](https://en.wikipedia.org/wiki/Tool-assisted_speedrun#Method).
  
  I'll investigate how such tools work and whats the idea behind them (and if indeed they do what I think they do). Will update!
  
  **Tags:** reinforcement-learning, machine-learning, neat, resource-request
  
  > **Comments:**
  > **username_1**: If you're asking for a tool, this question is off-topic. If you're looking for an approach/algorithm, then it could be on-topic. Questions that ask for software libraries should be asked on https://softwarerecs.stackexchange.com/ or DS SE. We focus on the theoretical/conceptual, philosophical and social aspects of AI (although we have had very few questions on the latter 2 topics recently). See https://ai.stackexchange.com/help/on-topic. Please, edit your post to clarify that you're not asking for a tool (unless that's an additional request), if you don't want your post to be closed.  
  > **username_2**: I don't think this is a tool as much as "why the process" sort of question.  The deepmind atari, afaict, didn't use the screen, but used a representation in memory.  Some versions (starcraft) can also look at other variables in memory like positions of objects.  
  > **username_0**: @username_2 How can you access the variables in memory and interact with the game without the game wasting time rendering?  
  > **username_2**: I think some of the answers have them. The screen is a pixel map. Rendering seeks to fill in all the pixels with intensity values. The internal representation that the computer holds would be very inefficient if it was all pixels. The JPEG image is discrete cosine coefficients. If you can expose the learner to the input to the renderer instead of its output, then you can give it a compressed form before that form is burdening the silicon.  
  > **username_3**: Note that graphics don't have to be rendered in real time and don't have to be rendered to the screen! You may be able to patch the game to render to a framebuffer and run as fast as possible (instead of slowing itself down to 100% normal speed)  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_4 on 2022-06-04. Score: 7
  
  It depends on the game environment and on the model being trained.
  
  * If you are training an agent that uses vision to decide action, then typically you need a copy of the rendered screen:
    + If that comes from a modern game using GPU acceleration to generate output, then you will likely need to leave that as is, and also transfer the rendered output back to the main memory, so it will likely be slower than running the game normally. There may be game settings that help improve speed.
    + If the output is from a game emulator for older games (such as Atari), then it probably won't use a GPU, and you will not need to spend time rendering to the screen. The emulator may be able to run in a mode that doesn't render, whilst still giving in-memory access to the screen buffer. It may also be able to run faster than real time. It will depend on emulator configuration settings that are available to you.
  * If the game is written "natively" in a simulator or semi-native environment like Pygame or Box2D, you should be able to disable rendering and save the time. This is how a lot of OpenAI Gym's environments are set up, and these do run a lot faster when not rendering - a simple flag you can set when starting the environment and/or starting an episode. You may even notice with these environments that putting the view window for the environment behind another window will speed things up considerably (but for automation it is more reliable to disable rendering).
  
  When the environments have been set up via a shared library, like Open AI's Gym, then the internal logic for deciding rendering (or not) will be different for each environment, but there may be a standardised config/method argument to determine whether the game engine is rendered to screen. That could apply to nearly all the environments available in the library. You should check the documentation if for example you are trying out NEAT on some Atari games that have been ready packaged for use with computer agents.
  
  > **Comments:**
  > **username_0**: Thanks for the detailed explanation! Talking about OpenAI, when they created openAI 5 for the "Dota 2" game, did they create a copy of it without rendering to allow fast training?  
  > **username_4**: @username_0: I don't know, but my understanding is that the RL environment for DoTA is not based on the pixels, but on the object models in the game via an API, which implies the kind of low-level access where that is possible.  
  
  ---
  
  ### Answer by username_5 on 2022-06-06. Score: 3
  
  ## Bypassing graphics
  
  As mentioned in username_4 username_4's answer, manipulating the engine to bypass graphics rendering to speed up AI simulation can be a valid approach. I have done that myself.
  
  But there are some caveats to consider/work around:
  
  * The graphics subsystem can be used for core gameplay programming as well. For example, the picking problem (Which object in the scene is the user selecting/clicking on?) [can be solved by rendering and evaluating framebuffers](http://www.opengl-tutorial.org/miscellaneous/clicking-on-objects/picking-with-an-opengl-hack/).
  * If the game has an FPS limiter, then you need to raise that FPS limit. Otherwise it quickly becomes the new bottleneck.
  * If the game also has real-time timers (i.e. wait five seconds to trigger some event), then your ultra-fast gameplay simulation can cause unintended effects/bugs. This is especially true if you break the aforementioned FPS limit.
  * Sometimes games can experience bugs and crashes when running at high framerates. ([Take this PCGamingWiki entry on
    *Peter Jackson's King Kong Gamer's Edition* as an example.](https://www.pcgamingwiki.com/wiki/Peter_Jackson%27s_King_Kong_Gamer%27s_Edition#Game_breaking_bugs_at_high_framerates)) With high framerates, gameplay simulation may operate with very small timesteps and glitches in physics and collision detection become more likely. Such effects will only intensify with ultra-high framerates from the accelerated AI simulation.
  
  ### DLL injection
  
  It is not uncommon for game mods to [hijack or replace DLLs](https://www.upguard.com/blog/dll-hijacking), including those of the graphics API. If you cannot manipulate the engine itself, then perhaps you can inject an alternative graphics API implementation that consists of stubs and mockups.
  
  ## Bypass whatever is irrelevant to gameplay
  
  Depending on the game and the goals of your project, you might take even more drastic steps to accelerate the simulation:
  
  * Bypass the audio layer as well. Besides graphics, audio is another major output channel that is often irrelevant to AI simulation.
  * Skip through cutscenes and dialogue, like a speedrunner would do.
  * If walking is not essential to gameplay, let the avatar teleport instead of simulating all the walking.
  * Skip animations in general. Always try to fast-forward to the "gameplay outcome" of an action.
  
  
  ---
  
  