Answers.Body: <blockquote>
  <p>Is it possible to have values of the states equal to 0 at the end of the value iteration?</p>
  </blockquote>
  <p>Yes.</p>
  <p>For a start, all terminal states should have a value of zero. This is not usually learned or calculated, but is by definition because the value represents the sum of expected future rewards and a terminal state should not have any. However, if the terminal states are implemented as &quot;absorbing&quot; states which always return 0 reward and do not transition away, then they can be learned as having a value of zero by e.g. value iteration. Caveat: This only works for a discounted return with <span class="math-container">$\gamma \lt 1$</span>.</p>
  <p>In addition, it is entirely possible to have an expected return of zero from any state. This might be composed of multiple positive and negative rewards that happen to sum to zero in expectation. A simple deterministic path through the state space that ended in a terminal state without receiving <em>any</em> positive or negative reward along the way would also have a value of zero.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-06-25T14:28:51.393Z
Answers.LastActivityDate: 2021-06-25T14:46:11.317Z
Answers.LastEditDate: 2021-06-25T14:46:11.317Z
Answers.UserDisplayName: Neil Slater
Body: <p>I am new to Reinforcement Learning and I am trying to self learn it. I have already posted some quesiton here and your answershave been really useful to me, so here I am posting another one.</p>
  <p>I am studying the value iteration, and while doing the simulation using python, I get that at some states it is associated a value of <span class="math-container">$0$</span>. I think I have to mention that I have tried to assign to the ststes an initial value different from zero, in order to simulate the fact that the agent already have some information about the enviroment before starting.</p>
  <p>So, my questio is:</p>
  <p><strong>Is it possible to have values of the states equal to <span class="math-container">$0$</span> at the end of the value iteration?</strong></p>
  
Tags: <reinforcement-learning><value-functions><value-iteration>
Title: Is it possible to have values of the states equal to $0$ at the end of the value iteration?
UserDisplayName: dcr
ThreadText: # Is it possible to have values of the states equal to $0$ at the end of the value iteration?
  
  (Asked by: username_0 on 2021-06-25)
  
  I am new to Reinforcement Learning and I am trying to self learn it. I have already posted some quesiton here and your answershave been really useful to me, so here I am posting another one.
  
  I am studying the value iteration, and while doing the simulation using python, I get that at some states it is associated a value of $0$. I think I have to mention that I have tried to assign to the ststes an initial value different from zero, in order to simulate the fact that the agent already have some information about the enviroment before starting.
  
  So, my questio is:
  
  **Is it possible to have values of the states equal to $0$ at the end of the value iteration?**
  
  **Tags:** reinforcement-learning, value-functions, value-iteration
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-06-25. Score: 2 | Accepted answer
  
  > Is it possible to have values of the states equal to 0 at the end of the value iteration?
  
  Yes.
  
  For a start, all terminal states should have a value of zero. This is not usually learned or calculated, but is by definition because the value represents the sum of expected future rewards and a terminal state should not have any. However, if the terminal states are implemented as "absorbing" states which always return 0 reward and do not transition away, then they can be learned as having a value of zero by e.g. value iteration. Caveat: This only works for a discounted return with $\gamma \lt 1$.
  
  In addition, it is entirely possible to have an expected return of zero from any state. This might be composed of multiple positive and negative rewards that happen to sum to zero in expectation. A simple deterministic path through the state space that ended in a terminal state without receiving *any* positive or negative reward along the way would also have a value of zero.
  
  
  ---
  
  