<p>It might seem to give the same update direction but would it converge to desirable policy parameters?</p>

<p>Actor-Critic is proposed alongside the policy gradient theorem in Sutton 1999. It is shown to maximize the state-value function. If you are able to show that the technique of yours is, in fact, maximizing some desirable objective function, you could propose it with some soundness as well.</p>

CC BY-SA 4.0
2019-02-27T09:13:36.550Z
2019-02-27T09:13:36.550Z
Phizaz
<p>I have read a lot on Actor Critic and I'm not convinced that there is a qualitative difference doing direct gradient updates on the network and slightly adjusting a soft-max output in the direction of the advantage function and doing gradient descent on the error. </p>

<p>Can anyone explain why updating the gradient directly is necessary? </p>

<reinforcement-learning><actor-critic-methods>
Why is gradient ascent necessary when training Actor Critic agents?
user6622581
# Why is gradient ascent necessary when training Actor Critic agents?

(Asked by: username_0 on 2019-02-18)

I have read a lot on Actor Critic and I'm not convinced that there is a qualitative difference doing direct gradient updates on the network and slightly adjusting a soft-max output in the direction of the advantage function and doing gradient descent on the error.

Can anyone explain why updating the gradient directly is necessary?

**Tags:** reinforcement-learning, actor-critic-methods


---

## 1 answers

### Answer by username_1 on 2019-02-27. Score: 1

It might seem to give the same update direction but would it converge to desirable policy parameters?

Actor-Critic is proposed alongside the policy gradient theorem in Sutton 1999. It is shown to maximize the state-value function. If you are able to show that the technique of yours is, in fact, maximizing some desirable objective function, you could propose it with some soundness as well.


---

