<p>In policy gradient theory subtracting state dependent baseline from Q(s,a) does not affect gradient of the objective function. I understand the proof shown below. One things is, if it's not affecting the gradient, then how this approach is better?
<a href="https://i.sstatic.net/N0XUY.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/N0XUY.png" alt="Proof that state dependent baseline does not affect gradient of the objective funtion" /></a></p>

CC BY-SA 4.0
2022-09-20T14:11:23.967Z
Short answer: variance reduction. Some details may be found, e.g., [here](https://danieltakeshi.github.io/2017/03/28/going-deeper-into-reinforcement-learning-fundamentals-of-policy-gradients/).
mikkola
CC BY-SA 4.0
2022-09-20T15:48:13.200Z
@mikkola thank you for your time end effort sir. I will check this out.
Davaasuren Nyamdavaa
<reinforcement-learning><policy-gradients>
In the policy gradient method, state dependent baseline does not affect gradient of the objective function. Then how this is better approach?
Davaasuren Nyamdavaa
# In the policy gradient method, state dependent baseline does not affect gradient of the objective function. Then how this is better approach?

(Asked by: username_0 on 2022-09-20)

In policy gradient theory subtracting state dependent baseline from Q(s,a) does not affect gradient of the objective function. I understand the proof shown below. One things is, if it's not affecting the gradient, then how this approach is better?
[![Proof that state dependent baseline does not affect gradient of the objective funtion](https://i.sstatic.net/N0XUY.png)](https://i.sstatic.net/N0XUY.png)

**Tags:** reinforcement-learning, policy-gradients

> **Comments:**
> **username_1**: Short answer: variance reduction. Some details may be found, e.g., [here](https://danieltakeshi.github.io/2017/03/28/going-deeper-into-reinforcement-learning-fundamentals-of-policy-gradients/).  
> **username_0**: @username_1 thank you for your time end effort sir. I will check this out.  

---

## 0 answers

