Answers.Body: <p>I think you are right. But then the next doubt would be <em>why</em> to include so many no-ops instead of just <em>original action space+ one no-op</em> space rather. And for this, I think it is to intentionally increase the chances for the agent to choose no-op action. And the reason to increase the chance might be to have some randomness or variety for the agent so that it is kind of better challenged to learn the correct actions.</p>
  <p>The accepted answer in the following link tells you the significance of having no-op actions that might help you to correlate and possibly deduce why that specific number of no-op actions in your paper exaample. <a href="https://stats.stackexchange.com/questions/351035/why-does-deep-q-learning-have-do-nothing-actions">https://stats.stackexchange.com/questions/351035/why-does-deep-q-learning-have-do-nothing-actions</a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-20T14:25:03.003Z
Answers.Text: Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer).
Answers.UserDisplayName: Community
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-06-20T14:19:53.603Z
Answers.LastActivityDate: 2022-06-20T14:35:27.657Z
Answers.LastEditDate: 2022-06-20T14:35:27.657Z
Answers.UserDisplayName: knowledge_seeker
Body: <p>I am just reading a paper titled &quot;Dueling Network Architectures for Deep Reinforcement Learning&quot;. In this paper, 4. Experiment (4.1 Policy evaluation), I just wonder how to form 5, 10, and 20 actions.</p>
  <p>According to this paper, the 10 and 20 action variants are formed by adding no-ops to the original. I can't understand this... Does it mean that actions are available: go up, go down, left, right, no-op, no-op, no-op... and no-op?</p>
  <p>Am I correct?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-06-20T14:24:31.513Z
Comments.Text: Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.
Comments.UserDisplayName: Community
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-06-21T19:10:28.157Z
Comments.Text: What do you mean by "to form 5, 10, and 20 actions"?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-06-23T07:14:07.887Z
Comments.Text: I just borrow that expression from the paper.
  I mean how to modify 5 action(up,down,left,right) in corridor env to 20 actions.
Comments.UserDisplayName: AIAIAIAIAIAI
Tags: <reinforcement-learning><dqn>
Title: How to form 10 and 20 actions in corridor environment, in the paper "Dueling Network Architectures for Deep Reinforcement Learning"?
UserDisplayName: AIAIAIAIAIAI
ThreadText: # How to form 10 and 20 actions in corridor environment, in the paper "Dueling Network Architectures for Deep Reinforcement Learning"?
  
  (Asked by: username_0 on 2022-06-20)
  
  I am just reading a paper titled "Dueling Network Architectures for Deep Reinforcement Learning". In this paper, 4. Experiment (4.1 Policy evaluation), I just wonder how to form 5, 10, and 20 actions.
  
  According to this paper, the 10 and 20 action variants are formed by adding no-ops to the original. I can't understand this... Does it mean that actions are available: go up, go down, left, right, no-op, no-op, no-op... and no-op?
  
  Am I correct?
  
  **Tags:** reinforcement-learning, dqn
  
  > **Comments:**
  > **username_1**: Please clarify your specific problem or provide additional details to highlight exactly what you need. As it's currently written, it's hard to tell exactly what you're asking.  
  > **username_2**: What do you mean by "to form 5, 10, and 20 actions"?  
  > **username_0**: I just borrow that expression from the paper. I mean how to modify 5 action(up,down,left,right) in corridor env to 20 actions.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2022-06-20. Score: 1
  
  I think you are right. But then the next doubt would be *why* to include so many no-ops instead of just *original action space+ one no-op* space rather. And for this, I think it is to intentionally increase the chances for the agent to choose no-op action. And the reason to increase the chance might be to have some randomness or variety for the agent so that it is kind of better challenged to learn the correct actions.
  
  The accepted answer in the following link tells you the significance of having no-op actions that might help you to correlate and possibly deduce why that specific number of no-op actions in your paper exaample. <https://stats.stackexchange.com/questions/351035/why-does-deep-q-learning-have-do-nothing-actions>
  
  > **Comments:**
  > **username_1**: Your answer could be improved with additional supporting information. Please [edit] to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers [in the help center](/help/how-to-answer).  
  
  ---
  
  