Body: <p>Lately, I have implemented <a href="https://arxiv.org/pdf/1312.5602.pdf" rel="nofollow noreferrer">DQN</a> for Atari Breakout. Here is the code: </p>
  
  <p><a href="https://github.com/JeremieGauthier/AI_Exercices/blob/master/Atari_Breakout/DQN_Breakout.py" rel="nofollow noreferrer">https://github.com/JeremieGauthier/AI_Exercices/blob/master/Atari_Breakout/DQN_Breakout.py</a> </p>
  
  <p>I have trained the agent for over 1500 episodes, but the training leveled off at around 5 as score. Could someone look at the code and point out a few things that should be corrected?</p>
  
  <p><a href="https://i.sstatic.net/hyvYK.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/hyvYK.png" alt="enter image description here"></a></p>
  
  <p>Actually, the training is not going more than 5 in average score. Is there a way to improve my performance?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-10T14:41:33.497Z
Comments.Text: Have you look at the original DQN paper? How long have they trained their network for?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-10T14:42:40.487Z
Comments.Text: Yeah, I have seen the original paper. I trained it for 3 to 5 hours
Comments.UserDisplayName: jgauth
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-10T14:43:54.547Z
Comments.Text: And how long have the original authors of DQN trained their network for? Probably a lot more than 3-5 hours. And in terms of episodes?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-10T14:55:04.117Z
Comments.Text: ''One epoch corresponds to 50000 minibatch weight updates or roughly 30 minutes of training time''. They achieved average reward of 1800, and in my case I get at most 5. Where am I wrong? Will I achieve 1600 or 1700  in average reward if I wait long enough. Right now, the code have some difficulties to just break through 5 in average reward
Comments.UserDisplayName: jgauth
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-10T14:58:26.553Z
Comments.Text: What is the batch size they used? You're using a batch size of `batch_size = 256`, according to your code. Just to make sure, are you using their exact configuration of the hyper-parameters?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-10T15:02:52.193Z
Comments.Text: @nbro It seems it is not indicated. Am I wrong? I tried to replicate the code as it is indicated
Comments.UserDisplayName: jgauth
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-10T15:36:41.437Z
Comments.Text: It's been a long time since I read that paper, but I was planning to read it again in the next days. Maybe I will have a look at it later.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-04-27T13:52:17.850Z
Comments.Text: Did the URL move? getting 404 from github.
  @jgauth
Comments.UserDisplayName: Yaniv Peretz
Tags: <reinforcement-learning><dqn><atari-games>
Title: Why isn't my DQN agent improving when trained on Atari Breakout?
UserDisplayName: jgauth
ThreadText: # Why isn't my DQN agent improving when trained on Atari Breakout?
  
  (Asked by: username_0 on 2020-04-10)
  
  Lately, I have implemented [DQN](https://arxiv.org/pdf/1312.5602.pdf) for Atari Breakout. Here is the code:
  
  <https://github.com/JeremieGauthier/AI_Exercices/blob/master/Atari_Breakout/DQN_Breakout.py>
  
  I have trained the agent for over 1500 episodes, but the training leveled off at around 5 as score. Could someone look at the code and point out a few things that should be corrected?
  
  [![enter image description here](https://i.sstatic.net/hyvYK.png)](https://i.sstatic.net/hyvYK.png)
  
  Actually, the training is not going more than 5 in average score. Is there a way to improve my performance?
  
  **Tags:** reinforcement-learning, dqn, atari-games
  > **Comments:**
  > **username_1**: Have you look at the original DQN paper? How long have they trained their network for?  
  > **username_0**: Yeah, I have seen the original paper. I trained it for 3 to 5 hours  
  > **username_1**: And how long have the original authors of DQN trained their network for? Probably a lot more than 3-5 hours. And in terms of episodes?  
  > **username_0**: ''One epoch corresponds to 50000 minibatch weight updates or roughly 30 minutes of training time''. They achieved average reward of 1800, and in my case I get at most 5. Where am I wrong? Will I achieve 1600 or 1700  in average reward if I wait long enough. Right now, the code have some difficulties to just break through 5 in average reward  
  > **username_1**: What is the batch size they used? You're using a batch size of `batch_size = 256`, according to your code. Just to make sure, are you using their exact configuration of the hyper-parameters?  
  > **username_0**: @nbro It seems it is not indicated. Am I wrong? I tried to replicate the code as it is indicated  
  > **username_1**: It's been a long time since I read that paper, but I was planning to read it again in the next days. Maybe I will have a look at it later.  
  > **username_2**: Did the URL move? getting 404 from github. @jgauth  
  
  ---
  
  ## 0 answers
  
  