Body: <p>An important property of a reinforcement learning problem is whether the environment of the agent is static, which means that nothing changes if the agent remains inactive. Different learning methods assume in varying degrees that the environment is static. </p>
  
  <p>How can I check if and (if so) where in the Monte Carlo algorithm, temporal difference learning (TD(0)), the Dyna-Q architecture, and R-Max a static environment is implicitly assumed? </p>
  
  <p>How could I modify the relevant learning methods so that they can in principle adapt to changing environments? (It can be assumed that <span class="math-container">$\epsilon$</span> is sufficiently large.)</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-06-17T19:39:25.973Z
Comments.Text: Hi and welcome to this community! Can you please tell us where you saw this definition of a "static environment". There are the notions of a _stochastic_ and _deterministic_ environments, which do not seem to be what you're refering to.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-06-24T20:55:38.200Z
Comments.Text: It seems to me that the default assumption is static, because, even if learning based on real-time, real-world data, the parameters are static. Thus, while the environment might change, the model being utilized is static..
Comments.UserDisplayName: DukeZhou
Tags: <reinforcement-learning><monte-carlo-methods><temporal-difference-methods><environment><dyna>
Title: How do I know if the assumption of a static environment is made?
UserDisplayName: maven
ThreadText: # How do I know if the assumption of a static environment is made?
  
  (Asked by: username_0 on 2019-06-17)
  
  An important property of a reinforcement learning problem is whether the environment of the agent is static, which means that nothing changes if the agent remains inactive. Different learning methods assume in varying degrees that the environment is static.
  
  How can I check if and (if so) where in the Monte Carlo algorithm, temporal difference learning (TD(0)), the Dyna-Q architecture, and R-Max a static environment is implicitly assumed?
  
  How could I modify the relevant learning methods so that they can in principle adapt to changing environments? (It can be assumed that $\epsilon$ is sufficiently large.)
  
  **Tags:** reinforcement-learning, monte-carlo-methods, temporal-difference-methods, environment, dyna
  
  > **Comments:**
  > **username_1**: Hi and welcome to this community! Can you please tell us where you saw this definition of a "static environment". There are the notions of a _stochastic_ and _deterministic_ environments, which do not seem to be what you're refering to.  
  > **username_2**: It seems to me that the default assumption is static, because, even if learning based on real-time, real-world data, the parameters are static. Thus, while the environment might change, the model being utilized is static..  
  
  ---
  
  ## 0 answers
  
  