<p>Yes, optimality is defined in the expectation of a random variable, variance is not taken in consideration</p>
<p>Consider 2 bandits, one that has <span class="math-container">$R^1 \sim N(0, 1)$</span> and the second one that has <span class="math-container">$R^2 \sim N(0,100)$</span> (<span class="math-container">$R$</span> being the reward function), then any policy <span class="math-container">$\pi$</span> will maximize the expected reward but for sure the greedy policy that always picks the first bandit has a lower variance than the greedy one that takes every time the second bandit</p>

CC BY-SA 4.0
2024-04-21T20:57:09.503Z
2024-04-21T20:57:09.503Z
Alberto
<p>If for an MDP there exist two optimal policies, it may be possible that the variance of their returns are different for a given state.</p>
<p>This is correct, right?</p>

<reinforcement-learning><markov-decision-process>
Optimality of two policies versus variance of returns from a state
DSPinfinity
# Optimality of two policies versus variance of returns from a state

(Asked by: username_0 on 2024-04-21)

If for an MDP there exist two optimal policies, it may be possible that the variance of their returns are different for a given state.

This is correct, right?

**Tags:** reinforcement-learning, markov-decision-process


---

## 1 answers

### Answer by username_1 on 2024-04-21. Score: 0 | Accepted answer

Yes, optimality is defined in the expectation of a random variable, variance is not taken in consideration

Consider 2 bandits, one that has $R^1 \sim N(0, 1)$ and the second one that has $R^2 \sim N(0,100)$ ($R$ being the reward function), then any policy $\pi$ will maximize the expected reward but for sure the greedy policy that always picks the first bandit has a lower variance than the greedy one that takes every time the second bandit


---

