Body: <p>In Policy Iteration (PI), the action generated by the policy, whether it's optimal or not w.r.t the current value function <span class="math-container">$v(s)$</span>. Whereas, in Value Iteration, the action is greedily generated w.r.t current <span class="math-container">$v(s)$</span>, which is an approximation of the <a href="https://en.wikipedia.org/wiki/Loss_function" rel="nofollow noreferrer">objective function</a> (as I understand). As a consequence, in the first few iterations, will Value Iteration perform better than Policy Iteration?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-09-11T17:52:14.390Z
Comments.Text: @DuttaA That's what I'm trying to figure out. I'm new to reinforcement learing. I'm used to notations in optimal control and classic optimization, where objective function is used. Since reinforcement learning is solving an optimization problem, I assume the value function will eventually converge to the objective function (or the optimal value function)? Then the greedy action w.r.t the optimal value function corresponds to the optimal control action that minimizes the objective function in optimal control.
Comments.UserDisplayName: qiang li
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-09-11T18:10:46.843Z
Comments.Text: @DuttaA I totally agree. I'm still taking the first steps to figure out what everything means in RL and their corresponding counterparts in my problems. After that, I'll confidently formulate my problems with terminology the RL community agreed on.
Comments.UserDisplayName: qiang li
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-09-13T22:44:23.677Z
Comments.Text: @qiangli In general, an objective function is just the function you want to optimize (either minimise or maximise). It seems that you're using the expression _objective function_ as a synonym for the function you want to learn or find.
Comments.UserDisplayName: nbro
Tags: <reinforcement-learning><comparison><value-iteration><policy-iteration>
Title: Is Value Iteration better than Policy Iteration for first few iterations?
UserDisplayName: qiang li
ThreadText: # Is Value Iteration better than Policy Iteration for first few iterations?
  
  (Asked by: username_0 on 2019-09-11)
  
  In Policy Iteration (PI), the action generated by the policy, whether it's optimal or not w.r.t the current value function $v(s)$. Whereas, in Value Iteration, the action is greedily generated w.r.t current $v(s)$, which is an approximation of the [objective function](https://en.wikipedia.org/wiki/Loss_function) (as I understand). As a consequence, in the first few iterations, will Value Iteration perform better than Policy Iteration?
  
  **Tags:** reinforcement-learning, comparison, value-iteration, policy-iteration
  
  > **Comments:**
  > **username_0**: @DuttaA That's what I'm trying to figure out. I'm new to reinforcement learing. I'm used to notations in optimal control and classic optimization, where objective function is used. Since reinforcement learning is solving an optimization problem, I assume the value function will eventually converge to the objective function (or the optimal value function)? Then the greedy action w.r.t the optimal value function corresponds to the optimal control action that minimizes the objective function in optimal control.  
  > **username_0**: @DuttaA I totally agree. I'm still taking the first steps to figure out what everything means in RL and their corresponding counterparts in my problems. After that, I'll confidently formulate my problems with terminology the RL community agreed on.  
  > **username_1**: @username_0 In general, an objective function is just the function you want to optimize (either minimise or maximise). It seems that you're using the expression _objective function_ as a synonym for the function you want to learn or find.  
  
  ---
  
  ## 0 answers
  
  