Body: <p>Given this pseudo code for value iteration:</p>
  <p><a href="https://i.sstatic.net/E5DFi.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/E5DFi.png" alt="enter image description here" /></a></p>
  <p>In the case of gamma=1, under what conditions on the MDP will we still be able to find the optimal policy?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-14T17:09:29.310Z
Comments.Text: The pseudocode as given is broken for $\gamma = 1$ (and technically for $\gamma = 0$, although that is more fixable) due to the condition in the repeat until loop. So could you clarify whether you really meant to ask about the specific pseudo-code given, or about convergence of value iteration in general when $\gamma=1$?
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-15T18:17:46.777Z
Comments.Text: @NeilSlater Hi, yes sorry, the pseducode is not defined for gamma = 1. I meant for value iteration in general. Would a condition on the MDP / environment that states that we always reach a final state, would mean that value iteration will give us the optimal policy?
Comments.UserDisplayName: Toffe1369
Tags: <markov-decision-process><convergence><value-iteration><discount-factor>
Title: Convergence of Value Iteration for Discount factor of 1
UserDisplayName: Toffe1369
ThreadText: # Convergence of Value Iteration for Discount factor of 1
  
  (Asked by: username_0 on 2023-01-14)
  
  Given this pseudo code for value iteration:
  
  [![enter image description here](https://i.sstatic.net/E5DFi.png)](https://i.sstatic.net/E5DFi.png)
  
  In the case of gamma=1, under what conditions on the MDP will we still be able to find the optimal policy?
  
  **Tags:** markov-decision-process, convergence, value-iteration, discount-factor
  > **Comments:**
  > **username_1**: The pseudocode as given is broken for $\gamma = 1$ (and technically for $\gamma = 0$, although that is more fixable) due to the condition in the repeat until loop. So could you clarify whether you really meant to ask about the specific pseudo-code given, or about convergence of value iteration in general when $\gamma=1$?  
  > **username_0**: @NeilSlater Hi, yes sorry, the pseducode is not defined for gamma = 1. I meant for value iteration in general. Would a condition on the MDP / environment that states that we always reach a final state, would mean that value iteration will give us the optimal policy?  
  
  ---
  
  ## 0 answers
  
  