<p>Let's say our task is to pick and place a block, like: <a href="https://gym.openai.com/envs/FetchPickAndPlace-v0/" rel="nofollow noreferrer">https://gym.openai.com/envs/FetchPickAndPlace-v0/</a></p>

<p>Reward function 1: -1 for block not placed, 0 for block placed</p>

<p>Reward function 2: 0 for block not placed, +1 for block placed</p>

<p>I noticed training 1 is much faster than 2... I am using the HER implementation from OpenAI. Why is that?</p>

CC BY-SA 4.0
2019-08-05T08:07:39.873Z
Could you please clarify when the rewards are assigned? It makes a big difference to most RL algorithms whether the "block not placed" reward is given on every time step or just at the end as an assssment.
Neil Slater
CC BY-SA 4.0
2019-08-05T12:07:49.313Z
And also, it seems that 1st option has a priori denser rewards. I'm not familiar with the HER implementation, apart from the experience replay, does anyone know if it centers the expected reward somehow? (State functions or GAE)
David
CC BY-SA 4.0
2019-08-07T04:31:21.733Z
These are given at each time step. The 'density' aka frequency of each type of reward is the same for each reward function.
user3180
<reinforcement-learning>
Why do these reward functions give different training curves?
user3180
# Why do these reward functions give different training curves?

(Asked by: username_0 on 2019-08-05)

Let's say our task is to pick and place a block, like: <https://gym.openai.com/envs/FetchPickAndPlace-v0/>

Reward function 1: -1 for block not placed, 0 for block placed

Reward function 2: 0 for block not placed, +1 for block placed

I noticed training 1 is much faster than 2... I am using the HER implementation from OpenAI. Why is that?

**Tags:** reinforcement-learning

> **Comments:**
> **username_1**: Could you please clarify when the rewards are assigned? It makes a big difference to most RL algorithms whether the "block not placed" reward is given on every time step or just at the end as an assssment.  
> **username_2**: And also, it seems that 1st option has a priori denser rewards. I'm not familiar with the HER implementation, apart from the experience replay, does anyone know if it centers the expected reward somehow? (State functions or GAE)  
> **username_0**: These are given at each time step. The 'density' aka frequency of each type of reward is the same for each reward function.  

---

## 0 answers

