Body: <p>I want to use a Deep Q-Network for a specific problem. My immediate rewards (<span class="math-container">$r_t = 0$</span>) are all zeros. But my terminal reward is a large positive value <span class="math-container">$(r_T=100$</span>). How could I normalize rewards to stabilize the training?  I think clipping rewards to be in range <span class="math-container">$[0,1]$</span> makes training harder because it just forces most values to be near zero.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-28T23:12:49.817Z
Comments.Text: why do you need to scale the reward here? the problem I would be concerned with is that almost all of your rewards are non-zero. sparse reward environments can be difficult to solve. perhaps you should look at the Hindsight Experience Replay algorithm as a starting point for overcoming sparse rewards.
Comments.UserDisplayName: David
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-29T07:51:47.137Z
Comments.Text: Thank you. I now understand what I should do. I need to Define a bigger state which contains smaller state and gradually decrease it's volume so at the end of training it is equal to smaller state.
Comments.UserDisplayName: bitWise
Tags: <reinforcement-learning>
Title: How to normalize rewards in DQN?
UserDisplayName: bitWise
ThreadText: # How to normalize rewards in DQN?
  
  (Asked by: username_0 on 2021-12-28)
  
  I want to use a Deep Q-Network for a specific problem. My immediate rewards ($r\_t = 0$) are all zeros. But my terminal reward is a large positive value $(r\_T=100$). How could I normalize rewards to stabilize the training? I think clipping rewards to be in range $[0,1]$ makes training harder because it just forces most values to be near zero.
  
  **Tags:** reinforcement-learning
  
  > **Comments:**
  > **username_1**: why do you need to scale the reward here? the problem I would be concerned with is that almost all of your rewards are non-zero. sparse reward environments can be difficult to solve. perhaps you should look at the Hindsight Experience Replay algorithm as a starting point for overcoming sparse rewards.  
  > **username_0**: Thank you. I now understand what I should do. I need to Define a bigger state which contains smaller state and gradually decrease it's volume so at the end of training it is equal to smaller state.  
  
  ---
  
  ## 0 answers
  
  