Answers.Body: <p>It seems that decaying the learning rate solved my problem. I changed <code>learning_rate</code> from <code>0.001</code> to <code>0.0001</code></p>
  
  <p><a href="https://i.sstatic.net/fWRJF.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/fWRJF.png" alt="enter image description here"></a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-15T15:22:17.420Z
Answers.Text: Hi jgauth, great that decreasing the learning rate has solved your issue. By "decaying the learning rate" @nbro most probably meant to decrease the learning rate **during training**. This means that instead of a fixed learning rate you use a function that takes the amount of passed training steps as input and outputs a learning rate.
Answers.UserDisplayName: Ray Walker
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-15T16:38:36.523Z
Answers.Text: @RayWalker Thanks, I did not notice that. It might be a good idea!
Answers.UserDisplayName: jgauth
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-14T22:45:01.710Z
Answers.LastActivityDate: 2020-04-14T22:45:01.710Z
Answers.UserDisplayName: jgauth
Body: <p><a href="https://i.sstatic.net/rzoYV.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/rzoYV.png" alt="enter image description here"></a></p>
  
  <p>The agent is trying to master the <a href="https://gym.openai.com/envs/Breakout-v0/" rel="nofollow noreferrer">Atari Breakout game</a>.</p>
  
  <p>Here is my <a href="https://colab.research.google.com/drive/1mXUkkq_CMm0CN_dUbjzuclAzVqjJxZIp" rel="nofollow noreferrer">code</a></p>
  
  <p>Is that normal that <code>reward_100</code> decreased that much after it hits 4.5? Is there a way to avoid that behavior?</p>
  
  <p>Be aware that <code>reward_100</code> is simply <code>mean_reward = np.mean(self.total_rewards[-100:])</code>. In other words, it is the mean over the last 100 rewards. On the graph, <code>reward_100</code> represents de y-axis and th number of episodes the x-axis. </p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-14T20:51:46.247Z
Comments.Text: These are just general questions about your ML approach. Are you decaying the learning rate? And how are you trading off exploitation and exploration?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-14T21:48:16.960Z
Comments.Text: Decaying the learning rate seems to be the right thing to do. It changed a lot of things.
Comments.UserDisplayName: jgauth
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-14T22:00:37.270Z
Comments.Text: Nice! Maybe you can provide a formal answer below, once you solve your problem ;)
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-15T15:13:15.423Z
Comments.Text: What algorithm are you using?
Comments.UserDisplayName: Ray Walker
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-15T15:17:26.723Z
Comments.Text: @RayWalker I used just a standard DQN. Look at my code in the question to know more info.
Comments.UserDisplayName: jgauth
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-04-15T15:25:43.307Z
Comments.Text: Thanks for the answer. It's good practice to include this information in the question. All the best to you.
Comments.UserDisplayName: Ray Walker
Tags: <reinforcement-learning><python><pytorch>
Title: Why are the rewards of my RL agent for the Atari Breakout game decreasing after a certain number of episodes?
UserDisplayName: jgauth
ThreadText: # Why are the rewards of my RL agent for the Atari Breakout game decreasing after a certain number of episodes?
  
  (Asked by: username_0 on 2020-04-14)
  
  [![enter image description here](https://i.sstatic.net/rzoYV.png)](https://i.sstatic.net/rzoYV.png)
  
  The agent is trying to master the [Atari Breakout game](https://gym.openai.com/envs/Breakout-v0/).
  
  Here is my [code](https://colab.research.google.com/drive/1mXUkkq_CMm0CN_dUbjzuclAzVqjJxZIp)
  
  Is that normal that `reward_100` decreased that much after it hits 4.5? Is there a way to avoid that behavior?
  
  Be aware that `reward_100` is simply `mean_reward = np.mean(self.total_rewards[-100:])`. In other words, it is the mean over the last 100 rewards. On the graph, `reward_100` represents de y-axis and th number of episodes the x-axis.
  
  **Tags:** reinforcement-learning, python, pytorch
  > **Comments:**
  > **username_1**: These are just general questions about your ML approach. Are you decaying the learning rate? And how are you trading off exploitation and exploration?  
  > **username_0**: Decaying the learning rate seems to be the right thing to do. It changed a lot of things.  
  > **username_1**: Nice! Maybe you can provide a formal answer below, once you solve your problem ;)  
  > **username_2**: What algorithm are you using?  
  > **username_0**: @RayWalker I used just a standard DQN. Look at my code in the question to know more info.  
  > **username_2**: Thanks for the answer. It's good practice to include this information in the question. All the best to you.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2020-04-14. Score: 1 | Accepted answer
  
  It seems that decaying the learning rate solved my problem. I changed `learning_rate` from `0.001` to `0.0001`
  
  [![enter image description here](https://i.sstatic.net/fWRJF.png)](https://i.sstatic.net/fWRJF.png)
  
  > **Comments:**
  > **username_2**: Hi jgauth, great that decreasing the learning rate has solved your issue. By "decaying the learning rate" @nbro most probably meant to decrease the learning rate **during training**. This means that instead of a fixed learning rate you use a function that takes the amount of passed training steps as input and outputs a learning rate.  
  > **username_0**: @RayWalker Thanks, I did not notice that. It might be a good idea!  
  
  ---
  
  