<p>Given this pseudo code for value iteration:</p>
<p><a href="https://i.sstatic.net/E5DFi.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/E5DFi.png" alt="enter image description here" /></a></p>
<p>In the case of gamma=1, under what conditions on the MDP will we still be able to find the optimal policy?</p>

CC BY-SA 4.0
2023-01-14T17:09:29.310Z
The pseudocode as given is broken for $\gamma = 1$ (and technically for $\gamma = 0$, although that is more fixable) due to the condition in the repeat until loop. So could you clarify whether you really meant to ask about the specific pseudo-code given, or about convergence of value iteration in general when $\gamma=1$?
Neil Slater
CC BY-SA 4.0
2023-01-15T18:17:46.777Z
@NeilSlater Hi, yes sorry, the pseducode is not defined for gamma = 1. I meant for value iteration in general. Would a condition on the MDP / environment that states that we always reach a final state, would mean that value iteration will give us the optimal policy?
Toffe1369
<markov-decision-process><convergence><value-iteration><discount-factor>
Convergence of Value Iteration for Discount factor of 1
Toffe1369
# Convergence of Value Iteration for Discount factor of 1

(Asked by: username_0 on 2023-01-14)

Given this pseudo code for value iteration:

[![enter image description here](https://i.sstatic.net/E5DFi.png)](https://i.sstatic.net/E5DFi.png)

In the case of gamma=1, under what conditions on the MDP will we still be able to find the optimal policy?

**Tags:** markov-decision-process, convergence, value-iteration, discount-factor

> **Comments:**
> **username_1**: The pseudocode as given is broken for $\gamma = 1$ (and technically for $\gamma = 0$, although that is more fixable) due to the condition in the repeat until loop. So could you clarify whether you really meant to ask about the specific pseudo-code given, or about convergence of value iteration in general when $\gamma=1$?  
> **username_0**: @NeilSlater Hi, yes sorry, the pseducode is not defined for gamma = 1. I meant for value iteration in general. Would a condition on the MDP / environment that states that we always reach a final state, would mean that value iteration will give us the optimal policy?  

---

## 0 answers

