Answers.Body: <p>Yes, optimality is defined in the expectation of a random variable, variance is not taken in consideration</p>
  <p>Consider 2 bandits, one that has <span class="math-container">$R^1 \sim N(0, 1)$</span> and the second one that has <span class="math-container">$R^2 \sim N(0,100)$</span> (<span class="math-container">$R$</span> being the reward function), then any policy <span class="math-container">$\pi$</span> will maximize the expected reward but for sure the greedy policy that always picks the first bandit has a lower variance than the greedy one that takes every time the second bandit</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-04-21T20:57:09.503Z
Answers.LastActivityDate: 2024-04-21T20:57:09.503Z
Answers.UserDisplayName: Alberto
Body: <p>If for an MDP there exist two optimal policies, it may be possible that the variance of their returns are different for a given state.</p>
  <p>This is correct, right?</p>
  
Tags: <reinforcement-learning><markov-decision-process>
Title: Optimality of two policies versus variance of returns from a state
UserDisplayName: DSPinfinity
ThreadText: # Optimality of two policies versus variance of returns from a state
  
  (Asked by: username_0 on 2024-04-21)
  
  If for an MDP there exist two optimal policies, it may be possible that the variance of their returns are different for a given state.
  
  This is correct, right?
  
  **Tags:** reinforcement-learning, markov-decision-process
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-04-21. Score: 0 | Accepted answer
  
  Yes, optimality is defined in the expectation of a random variable, variance is not taken in consideration
  
  Consider 2 bandits, one that has $R^1 \sim N(0, 1)$ and the second one that has $R^2 \sim N(0,100)$ ($R$ being the reward function), then any policy $\pi$ will maximize the expected reward but for sure the greedy policy that always picks the first bandit has a lower variance than the greedy one that takes every time the second bandit
  
  
  ---
  
  