Body: <p>Are there any algorithms to use reinforcement learning to learn optimal policies in partially observable Markov decision process (POMDP) i.e. when the state is not perfectly observed? More specifically, how does one update the belief state using Bayes' rule when the update Q kernel is not known?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-10-08T00:29:18.247Z
Comments.Text: What do you mean by "update Q kernel"?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-10T22:26:31.117Z
Comments.Text: So a POMDP has a state update kernel Q such that $x_{t+1} \sim Q(\cdot|x_t,a_t)$ and an observation kernel such that the observation $y_t\sim Q^o(\cdot|x_t)$. By update I mean the former, state update kernel
Comments.UserDisplayName: Deepanshu Vasal
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-10T22:31:06.903Z
Comments.Text: Can you please link me to a research paper or book (or whatever) that uses the word "kernel"? Maybe this is only a terminology issue, but I don't think that "kernel" is the right word here.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-12T00:28:01.907Z
Comments.Text: Here is a paper [link](http://www.ams.sunysb.edu/~feinberg/public/Feinberg_KZgMOR.pdf). In general you can google pomdp transition kernel to get many such results
Comments.UserDisplayName: Deepanshu Vasal
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-19T18:53:30.580Z
Comments.Text: I am not sure if this is a duplicate of [this one](https://ai.stackexchange.com/q/11612/2444). I don't think so, as you're also asking a more specific question, but maybe you were just curious about how one could apply RL to POMDPs. Let me know.
Comments.UserDisplayName: nbro
Tags: <reinforcement-learning><reference-request><pomdp>
Title: Is there a way to do reinforcement learning in POMDP?
UserDisplayName: Deepanshu Vasal
ThreadText: # Is there a way to do reinforcement learning in POMDP?
  
  (Asked by: username_0 on 2019-10-06)
  
  Are there any algorithms to use reinforcement learning to learn optimal policies in partially observable Markov decision process (POMDP) i.e. when the state is not perfectly observed? More specifically, how does one update the belief state using Bayes' rule when the update Q kernel is not known?
  
  **Tags:** reinforcement-learning, reference-request, pomdp
  
  > **Comments:**
  > **username_1**: What do you mean by "update Q kernel"?  
  > **username_0**: So a POMDP has a state update kernel Q such that $x_{t+1} \sim Q(\cdot|x_t,a_t)$ and an observation kernel such that the observation $y_t\sim Q^o(\cdot|x_t)$. By update I mean the former, state update kernel  
  > **username_1**: Can you please link me to a research paper or book (or whatever) that uses the word "kernel"? Maybe this is only a terminology issue, but I don't think that "kernel" is the right word here.  
  > **username_0**: Here is a paper [link](http://www.ams.sunysb.edu/~feinberg/public/Feinberg_KZgMOR.pdf). In general you can google pomdp transition kernel to get many such results  
  > **username_1**: I am not sure if this is a duplicate of [this one](https://ai.stackexchange.com/q/11612/2444). I don't think so, as you're also asking a more specific question, but maybe you were just curious about how one could apply RL to POMDPs. Let me know.  
  
  ---
  
  ## 0 answers
  
  