Answers.Body: <p>Thompson Sampling (TS) is used in the context of <a href="https://ai.stackexchange.com/a/20861/2444">bandits</a>, which is a special case of the RL problem.</p>
  <p>You can also use TS for the full RL problem, but that can lead to <strong>inefficient exploration</strong>. To know more about this issue, you could read</p>
  <ul>
  <li>the section <a href="https://web.stanford.edu/%7Ebvr/pubs/TS_Tutorial.pdf#page=62" rel="nofollow noreferrer"><strong>7.5 Reinforcement Learning in Markov Decision Processes</strong></a> (p. 62) of the tutorial <a href="https://web.stanford.edu/%7Ebvr/pubs/TS_Tutorial.pdf" rel="nofollow noreferrer">A Tutorial on Thompson Sampling</a> (2017) by Russo et al.,</li>
  <li><a href="https://ai.stackexchange.com/a/24631/2444">my answer here</a>, and</li>
  <li>the paper <a href="https://arxiv.org/pdf/1703.07608.pdf" rel="nofollow noreferrer">Deep Exploration via Randomized Value Functions</a> (2019, JMLR) by Ian Osband et al.</li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-12-05T18:08:20.737Z
Answers.LastActivityDate: 2021-12-05T18:08:20.737Z
Answers.UserDisplayName: nbro
Body: <p>I often see Thompson Sampling in RL literature, however, I am not able to relate it to any of the current RL techniques. How exactly does it fit with RL?</p>
  
Tags: <reinforcement-learning><thompson-sampling>
Title: Why is Thompson Sampling considered a part of Reinforcement Learning?
UserDisplayName: desert_ranger
ThreadText: # Why is Thompson Sampling considered a part of Reinforcement Learning?
  
  (Asked by: username_0 on 2021-12-05)
  
  I often see Thompson Sampling in RL literature, however, I am not able to relate it to any of the current RL techniques. How exactly does it fit with RL?
  
  **Tags:** reinforcement-learning, thompson-sampling
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-12-05. Score: 3 | Accepted answer
  
  Thompson Sampling (TS) is used in the context of [bandits](https://ai.stackexchange.com/a/20861/2444), which is a special case of the RL problem.
  
  You can also use TS for the full RL problem, but that can lead to **inefficient exploration**. To know more about this issue, you could read
  
  * the section [**7.5 Reinforcement Learning in Markov Decision Processes**](https://web.stanford.edu/%7Ebvr/pubs/TS_Tutorial.pdf#page=62) (p. 62) of the tutorial [A Tutorial on Thompson Sampling](https://web.stanford.edu/%7Ebvr/pubs/TS_Tutorial.pdf) (2017) by Russo et al.,
  * [my answer here](https://ai.stackexchange.com/a/24631/2444), and
  * the paper [Deep Exploration via Randomized Value Functions](https://arxiv.org/pdf/1703.07608.pdf) (2019, JMLR) by Ian Osband et al.
  
  
  ---
  
  