<p>I want to use a Deep Q-Network for a specific problem. My immediate rewards (<span class="math-container">$r_t = 0$</span>) are all zeros. But my terminal reward is a large positive value <span class="math-container">$(r_T=100$</span>). How could I normalize rewards to stabilize the training?  I think clipping rewards to be in range <span class="math-container">$[0,1]$</span> makes training harder because it just forces most values to be near zero.</p>

CC BY-SA 4.0
2021-12-28T23:12:49.817Z
why do you need to scale the reward here? the problem I would be concerned with is that almost all of your rewards are non-zero. sparse reward environments can be difficult to solve. perhaps you should look at the Hindsight Experience Replay algorithm as a starting point for overcoming sparse rewards.
David
CC BY-SA 4.0
2021-12-29T07:51:47.137Z
Thank you. I now understand what I should do. I need to Define a bigger state which contains smaller state and gradually decrease it's volume so at the end of training it is equal to smaller state.
bitWise
<reinforcement-learning>
How to normalize rewards in DQN?
bitWise
# How to normalize rewards in DQN?

(Asked by: username_0 on 2021-12-28)

I want to use a Deep Q-Network for a specific problem. My immediate rewards ($r\_t = 0$) are all zeros. But my terminal reward is a large positive value $(r\_T=100$). How could I normalize rewards to stabilize the training? I think clipping rewards to be in range $[0,1]$ makes training harder because it just forces most values to be near zero.

**Tags:** reinforcement-learning

> **Comments:**
> **username_1**: why do you need to scale the reward here? the problem I would be concerned with is that almost all of your rewards are non-zero. sparse reward environments can be difficult to solve. perhaps you should look at the Hindsight Experience Replay algorithm as a starting point for overcoming sparse rewards.  
> **username_0**: Thank you. I now understand what I should do. I need to Define a bigger state which contains smaller state and gradually decrease it's volume so at the end of training it is equal to smaller state.  

---

## 0 answers

