Body: <p>Suppose I learn an optimal policy <span class="math-container">$\pi(a|c)$</span> for a contextual multi-armed bandit problem, where the context <span class="math-container">$c$</span> is a composite of multiple context variables <span class="math-container">$c = c_1, c_2, c_3$</span>. For example, the context is specified by three Bernoulli variables. </p>
  
  <p>Is there any literature on how to determine the optimal policy in the event where I no longer have access to one of the context variables? </p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-11-27T05:21:03.523Z
Comments.Text: Hello new user. Maybe explaining how the context variable is lost would help, the question become more clear.
Comments.UserDisplayName: Michael Hearn
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-11-27T13:55:49.883Z
Comments.Text: So, for example, imagine you have a robot who learns to take actions based on inputs which are three bernoulli variables. We then ast it to select an action but only provide the first two variables. For example, its sense for the third variable breaks. Should we marginalize over the expected reward w.r.t the third input variable?
Comments.UserDisplayName: user31663
Tags: <reinforcement-learning><markov-decision-process><multi-armed-bandits><contextual-bandits>
Title: How do I determine the optimal policy in a bandit problem with missing contexts?
UserDisplayName: user31663
ThreadText: # How do I determine the optimal policy in a bandit problem with missing contexts?
  
  (Asked by: username_0 on 2019-11-26)
  
  Suppose I learn an optimal policy $\pi(a|c)$ for a contextual multi-armed bandit problem, where the context $c$ is a composite of multiple context variables $c = c\_1, c\_2, c\_3$. For example, the context is specified by three Bernoulli variables.
  
  Is there any literature on how to determine the optimal policy in the event where I no longer have access to one of the context variables?
  
  **Tags:** reinforcement-learning, markov-decision-process, multi-armed-bandits, contextual-bandits
  
  > **Comments:**
  > **username_1**: Hello new user. Maybe explaining how the context variable is lost would help, the question become more clear.  
  > **username_0**: So, for example, imagine you have a robot who learns to take actions based on inputs which are three bernoulli variables. We then ast it to select an action but only provide the first two variables. For example, its sense for the third variable breaks. Should we marginalize over the expected reward w.r.t the third input variable?  
  
  ---
  
  ## 0 answers
  
  