Body: <p>I am trying to build a DQN model for the Atari Pong game, but I am not sure whether the model is learning at all.</p>
  
  <p>I am using the architecture described in the paper <a href="https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf" rel="nofollow noreferrer">Playing Atari with Deep Reinforcement Learning</a>. And I tested the model on a simpler environment (like CartPole), which worked great, but I am not seeing any progress at all with Pong, I have been training the model for 2-3 hours and its performance is no better than taking random actions.</p>
  
  <p>Should I just keep waiting or there might be something wrong with my code. Around how many episodes should it take before I see some positive results?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-01T13:56:14.700Z
Comments.Text: Don't bother with hours, everyone has different hardware. They say in paper that they train for 10 million frames which is 10 million timesteps in the environment. How many did you complete in 2-3 hours ?
Comments.UserDisplayName: Brale
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-01T14:11:06.850Z
Comments.Text: @Brale_ 100 episodes takes roughly 2 hours, and each episode takes maximum 10,000 steps (I am running each episode until game is lost/won).
Comments.UserDisplayName: Ach113
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-01T14:32:38.413Z
Comments.Text: I don't expect same results as in paper in just 100 episodes, but I should be getting at least some improvements right?
Comments.UserDisplayName: Ach113
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-01T14:34:24.337Z
Comments.Text: Well for you it could take entire day to reach around 10 million timesteps. Leave it over night for 7-8 hours and see in the morning if it gets any better. If not then you might have some bug.
Comments.UserDisplayName: Brale
Tags: <reinforcement-learning><dqn><deep-rl><implementation><computational-complexity>
Title: How much time does it take to train DQN on Atari environment?
UserDisplayName: Ach113
ThreadText: # How much time does it take to train DQN on Atari environment?
  
  (Asked by: username_0 on 2020-04-01)
  
  I am trying to build a DQN model for the Atari Pong game, but I am not sure whether the model is learning at all.
  
  I am using the architecture described in the paper [Playing Atari with Deep Reinforcement Learning](https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf). And I tested the model on a simpler environment (like CartPole), which worked great, but I am not seeing any progress at all with Pong, I have been training the model for 2-3 hours and its performance is no better than taking random actions.
  
  Should I just keep waiting or there might be something wrong with my code. Around how many episodes should it take before I see some positive results?
  
  **Tags:** reinforcement-learning, dqn, deep-rl, implementation, computational-complexity
  
  > **Comments:**
  > **username_1**: Don't bother with hours, everyone has different hardware. They say in paper that they train for 10 million frames which is 10 million timesteps in the environment. How many did you complete in 2-3 hours ?  
  > **username_0**: @Brale_ 100 episodes takes roughly 2 hours, and each episode takes maximum 10,000 steps (I am running each episode until game is lost/won).  
  > **username_0**: I don't expect same results as in paper in just 100 episodes, but I should be getting at least some improvements right?  
  > **username_1**: Well for you it could take entire day to reach around 10 million timesteps. Leave it over night for 7-8 hours and see in the morning if it gets any better. If not then you might have some bug.  
  
  ---
  
  ## 0 answers
  
  