<p>Thompson Sampling (TS) is used in the context of <a href="https://ai.stackexchange.com/a/20861/2444">bandits</a>, which is a special case of the RL problem.</p>
<p>You can also use TS for the full RL problem, but that can lead to <strong>inefficient exploration</strong>. To know more about this issue, you could read</p>
<ul>
<li>the section <a href="https://web.stanford.edu/%7Ebvr/pubs/TS_Tutorial.pdf#page=62" rel="nofollow noreferrer"><strong>7.5 Reinforcement Learning in Markov Decision Processes</strong></a> (p. 62) of the tutorial <a href="https://web.stanford.edu/%7Ebvr/pubs/TS_Tutorial.pdf" rel="nofollow noreferrer">A Tutorial on Thompson Sampling</a> (2017) by Russo et al.,</li>
<li><a href="https://ai.stackexchange.com/a/24631/2444">my answer here</a>, and</li>
<li>the paper <a href="https://arxiv.org/pdf/1703.07608.pdf" rel="nofollow noreferrer">Deep Exploration via Randomized Value Functions</a> (2019, JMLR) by Ian Osband et al.</li>
</ul>

CC BY-SA 4.0
2021-12-05T18:08:20.737Z
2021-12-05T18:08:20.737Z
nbro
<p>I often see Thompson Sampling in RL literature, however, I am not able to relate it to any of the current RL techniques. How exactly does it fit with RL?</p>

<reinforcement-learning><thompson-sampling>
Why is Thompson Sampling considered a part of Reinforcement Learning?
desert_ranger
# Why is Thompson Sampling considered a part of Reinforcement Learning?

(Asked by: username_0 on 2021-12-05)

I often see Thompson Sampling in RL literature, however, I am not able to relate it to any of the current RL techniques. How exactly does it fit with RL?

**Tags:** reinforcement-learning, thompson-sampling


---

## 1 answers

### Answer by username_1 on 2021-12-05. Score: 3 | Accepted answer

Thompson Sampling (TS) is used in the context of [bandits](https://ai.stackexchange.com/a/20861/2444), which is a special case of the RL problem.

You can also use TS for the full RL problem, but that can lead to **inefficient exploration**. To know more about this issue, you could read

* the section [**7.5 Reinforcement Learning in Markov Decision Processes**](https://web.stanford.edu/%7Ebvr/pubs/TS_Tutorial.pdf#page=62) (p. 62) of the tutorial [A Tutorial on Thompson Sampling](https://web.stanford.edu/%7Ebvr/pubs/TS_Tutorial.pdf) (2017) by Russo et al.,
* [my answer here](https://ai.stackexchange.com/a/24631/2444), and
* the paper [Deep Exploration via Randomized Value Functions](https://arxiv.org/pdf/1703.07608.pdf) (2019, JMLR) by Ian Osband et al.


---

