<p>Are there any algorithms to use reinforcement learning to learn optimal policies in partially observable Markov decision process (POMDP) i.e. when the state is not perfectly observed? More specifically, how does one update the belief state using Bayes' rule when the update Q kernel is not known?</p>

CC BY-SA 4.0
2019-10-08T00:29:18.247Z
What do you mean by "update Q kernel"?
nbro
CC BY-SA 4.0
2021-01-10T22:26:31.117Z
So a POMDP has a state update kernel Q such that $x_{t+1} \sim Q(\cdot|x_t,a_t)$ and an observation kernel such that the observation $y_t\sim Q^o(\cdot|x_t)$. By update I mean the former, state update kernel
Deepanshu Vasal
CC BY-SA 4.0
2021-01-10T22:31:06.903Z
Can you please link me to a research paper or book (or whatever) that uses the word "kernel"? Maybe this is only a terminology issue, but I don't think that "kernel" is the right word here.
nbro
CC BY-SA 4.0
2021-01-12T00:28:01.907Z
Here is a paper [link](http://www.ams.sunysb.edu/~feinberg/public/Feinberg_KZgMOR.pdf). In general you can google pomdp transition kernel to get many such results
Deepanshu Vasal
CC BY-SA 4.0
2021-12-19T18:53:30.580Z
I am not sure if this is a duplicate of [this one](https://ai.stackexchange.com/q/11612/2444). I don't think so, as you're also asking a more specific question, but maybe you were just curious about how one could apply RL to POMDPs. Let me know.
nbro
<reinforcement-learning><reference-request><pomdp>
Is there a way to do reinforcement learning in POMDP?
Deepanshu Vasal
# Is there a way to do reinforcement learning in POMDP?

(Asked by: username_0 on 2019-10-06)

Are there any algorithms to use reinforcement learning to learn optimal policies in partially observable Markov decision process (POMDP) i.e. when the state is not perfectly observed? More specifically, how does one update the belief state using Bayes' rule when the update Q kernel is not known?

**Tags:** reinforcement-learning, reference-request, pomdp

> **Comments:**
> **username_1**: What do you mean by "update Q kernel"?  
> **username_0**: So a POMDP has a state update kernel Q such that $x_{t+1} \sim Q(\cdot|x_t,a_t)$ and an observation kernel such that the observation $y_t\sim Q^o(\cdot|x_t)$. By update I mean the former, state update kernel  
> **username_1**: Can you please link me to a research paper or book (or whatever) that uses the word "kernel"? Maybe this is only a terminology issue, but I don't think that "kernel" is the right word here.  
> **username_0**: Here is a paper [link](http://www.ams.sunysb.edu/~feinberg/public/Feinberg_KZgMOR.pdf). In general you can google pomdp transition kernel to get many such results  
> **username_1**: I am not sure if this is a duplicate of [this one](https://ai.stackexchange.com/q/11612/2444). I don't think so, as you're also asking a more specific question, but maybe you were just curious about how one could apply RL to POMDPs. Let me know.  

---

## 0 answers

