Answers.Body: <p>Based on <a href="https://github.com/openai/baselines/issues/240#issuecomment-391165056" rel="nofollow noreferrer">this comment</a> in the Issue I created about this question on github, it looks like there is confirmation that at least DeepMind does <strong>not</strong> use this kind of functionality in their Atari experiments, contrary to what is implied by the comments in the OpenAI baselines code.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2018-05-23T08:46:19.700Z
Answers.LastActivityDate: 2018-05-23T08:46:19.700Z
Answers.UserDisplayName: Dennis Soemers
Body: <p>In some Atari games in the Arcade Learning Environment (ALE), it is necessary to press <code>FIRE</code> once to start a game. Because it may be difficult for a Reinforcement Learning (RL) agent to learn this, they may often waste a lot of time executing actions that do nothing. Therefore, I get the impression that some people hardcode their agent to press that <code>FIRE</code> button once when necessary. </p>
  
  <p>For example, in OpenAI's <a href="https://github.com/openai/baselines" rel="nofollow noreferrer">baselines repository</a>, this is implemented using the <a href="https://github.com/openai/baselines/blob/master/baselines/common/atari_wrappers.py#L37" rel="nofollow noreferrer"><code>FireResetEnv</code></a> wrapper. Further down, in their <a href="https://github.com/openai/baselines/blob/master/baselines/common/atari_wrappers.py#L223" rel="nofollow noreferrer"><code>wrap_deepmind</code></a> (which applies that wrapper among others), it is implied that DeepMind tends to use this functionality in all of their publications. I have not been able to find a reference for this claim though.</p>
  
  <hr>
  
  <p><strong>My question is</strong>: is it common in published research (by DeepMind or others) to use the functionality described above? I'd say that, if this is the case, it should be explicitly mentioned in these papers (because it's important to know if hardcoded domain knowledge was added to a learning agent), but I have been unable to explicitly find this after looking through a wide variety of papers. So, based on this, I'd be inclined to believe the answer is "no". The main thing that confuses me then is the implication (without reference) in the OpenAI baselines repository that the answer would be "yes".</p>
  
Tags: <reinforcement-learning><research><deepmind>
Title: Is it common in RL research with Atari/ALE to automatically press FIRE to start games?
UserDisplayName: Dennis Soemers
ThreadText: # Is it common in RL research with Atari/ALE to automatically press FIRE to start games?
  
  (Asked by: username_0 on 2018-02-22)
  
  In some Atari games in the Arcade Learning Environment (ALE), it is necessary to press `FIRE` once to start a game. Because it may be difficult for a Reinforcement Learning (RL) agent to learn this, they may often waste a lot of time executing actions that do nothing. Therefore, I get the impression that some people hardcode their agent to press that `FIRE` button once when necessary.
  
  For example, in OpenAI's [baselines repository](https://github.com/openai/baselines), this is implemented using the [`FireResetEnv`](https://github.com/openai/baselines/blob/master/baselines/common/atari_wrappers.py#L37) wrapper. Further down, in their [`wrap_deepmind`](https://github.com/openai/baselines/blob/master/baselines/common/atari_wrappers.py#L223) (which applies that wrapper among others), it is implied that DeepMind tends to use this functionality in all of their publications. I have not been able to find a reference for this claim though.
  
  ---
  
  **My question is**: is it common in published research (by DeepMind or others) to use the functionality described above? I'd say that, if this is the case, it should be explicitly mentioned in these papers (because it's important to know if hardcoded domain knowledge was added to a learning agent), but I have been unable to explicitly find this after looking through a wide variety of papers. So, based on this, I'd be inclined to believe the answer is "no". The main thing that confuses me then is the implication (without reference) in the OpenAI baselines repository that the answer would be "yes".
  
  **Tags:** reinforcement-learning, research, deepmind
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2018-05-23. Score: 4 | Accepted answer
  
  Based on [this comment](https://github.com/openai/baselines/issues/240#issuecomment-391165056) in the Issue I created about this question on github, it looks like there is confirmation that at least DeepMind does **not** use this kind of functionality in their Atari experiments, contrary to what is implied by the comments in the OpenAI baselines code.
  
  
  ---
  
  