Body: <p>Currently, I am reading into the Multi-Armed-Bandit problem and found the special case of non-stationary (environment and its attributes, like the reward distribution, change over time) stochastic environments. Since this is an adversarial MAB problem, no context is available at the moment. I've read that <span class="math-container">$\epsilon$</span>-greedy, Exp3 and the FPL algorithm work. But some tutorials like TensorFlow ones use LinUCB and other algorithms, which are never mentioned in any papers.</p>
  <p>So, my question is, basically: which algorithms work on the non-stationary stochastic environments?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-01-23T18:09:57.943Z
Comments.Text: What do you mean by "LinUCB and other algorithms, which are never mentioned in any papers."? I've seen LinUCB being mentioned in several papers that I've come across on bandits. So, which papers do you have in mind? Do you mean papers related to non-stationary environments? Also, can you clarify this "Since this is an adversarial MAB problem, no context is available at the moment."? Are you referring to the "context" as in "contextual bandits"? I don't see the connection between being "an adversarial MAB problem" and not having access to "context" (whatever you mean by this).
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-01-24T10:21:16.347Z
Comments.Text: Hi, yes I've been unclear in this points. By "LinUCB and other algorithms, which are never mentioned in any papers." I do mean papers related to non-stationary environments. And by "no context" I mean the base case of the MAB problem, so without any context like in contextual bandits. I just want to make the example simpler to understand it better. Thats why I am excluding the context. I hope this made it clearer.
Comments.UserDisplayName: paperplan3
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-01-24T11:18:26.700Z
Comments.Text: Thanks for the clarifications. Can you please [include this info directly into your post](https://ai.stackexchange.com/posts/34259/edit)? I would also recommend that you provide the link to 1-2 papers that you came across on non-stationary environments.
Comments.UserDisplayName: nbro
Tags: <reinforcement-learning><tensorflow><environment><multi-armed-bandits>
Title: Which algorithms work in a non-stationary stochastic environment?
UserDisplayName: paperplan3
ThreadText: # Which algorithms work in a non-stationary stochastic environment?
  
  (Asked by: username_0 on 2022-01-23)
  
  Currently, I am reading into the Multi-Armed-Bandit problem and found the special case of non-stationary (environment and its attributes, like the reward distribution, change over time) stochastic environments. Since this is an adversarial MAB problem, no context is available at the moment. I've read that $\epsilon$-greedy, Exp3 and the FPL algorithm work. But some tutorials like TensorFlow ones use LinUCB and other algorithms, which are never mentioned in any papers.
  
  So, my question is, basically: which algorithms work on the non-stationary stochastic environments?
  
  **Tags:** reinforcement-learning, tensorflow, environment, multi-armed-bandits
  
  > **Comments:**
  > **username_1**: What do you mean by "LinUCB and other algorithms, which are never mentioned in any papers."? I've seen LinUCB being mentioned in several papers that I've come across on bandits. So, which papers do you have in mind? Do you mean papers related to non-stationary environments? Also, can you clarify this "Since this is an adversarial MAB problem, no context is available at the moment."? Are you referring to the "context" as in "contextual bandits"? I don't see the connection between being "an adversarial MAB problem" and not having access to "context" (whatever you mean by this).  
  > **username_0**: Hi, yes I've been unclear in this points. By "LinUCB and other algorithms, which are never mentioned in any papers." I do mean papers related to non-stationary environments. And by "no context" I mean the base case of the MAB problem, so without any context like in contextual bandits. I just want to make the example simpler to understand it better. Thats why I am excluding the context. I hope this made it clearer.  
  > **username_1**: Thanks for the clarifications. Can you please [include this info directly into your post](https://ai.stackexchange.com/posts/34259/edit)? I would also recommend that you provide the link to 1-2 papers that you came across on non-stationary environments.  
  
  ---
  
  ## 0 answers
  
  