Answers.Body: <p>It might seem to give the same update direction but would it converge to desirable policy parameters?</p>
  
  <p>Actor-Critic is proposed alongside the policy gradient theorem in Sutton 1999. It is shown to maximize the state-value function. If you are able to show that the technique of yours is, in fact, maximizing some desirable objective function, you could propose it with some soundness as well.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-02-27T09:13:36.550Z
Answers.LastActivityDate: 2019-02-27T09:13:36.550Z
Answers.UserDisplayName: Phizaz
Body: <p>I have read a lot on Actor Critic and I'm not convinced that there is a qualitative difference doing direct gradient updates on the network and slightly adjusting a soft-max output in the direction of the advantage function and doing gradient descent on the error. </p>
  
  <p>Can anyone explain why updating the gradient directly is necessary? </p>
  
Tags: <reinforcement-learning><actor-critic-methods>
Title: Why is gradient ascent necessary when training Actor Critic agents?
UserDisplayName: user6622581
ThreadText: # Why is gradient ascent necessary when training Actor Critic agents?
  
  (Asked by: username_0 on 2019-02-18)
  
  I have read a lot on Actor Critic and I'm not convinced that there is a qualitative difference doing direct gradient updates on the network and slightly adjusting a soft-max output in the direction of the advantage function and doing gradient descent on the error.
  
  Can anyone explain why updating the gradient directly is necessary?
  
  **Tags:** reinforcement-learning, actor-critic-methods
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-02-27. Score: 1
  
  It might seem to give the same update direction but would it converge to desirable policy parameters?
  
  Actor-Critic is proposed alongside the policy gradient theorem in Sutton 1999. It is shown to maximize the state-value function. If you are able to show that the technique of yours is, in fact, maximizing some desirable objective function, you could propose it with some soundness as well.
  
  
  ---
  
  