Answers.Body: <p>I don't recommend changing the rules of the environment.</p>
  <p>What you could do:</p>
  <p>Perform a method called bucketing i.e. take a value from a continuous state space see which discrete bucket it should go into and then let your agent use the bucket number as the observation.</p>
  <p>e.g. Say I do have a continuous state space with one variable in range <span class="math-container">$[-\infty,\infty]$</span></p>
  <p>The buckets can be as follows:</p>
  <p>0). x &lt; -1000</p>
  <p>1). -1000 <span class="math-container">$\le$</span> x <span class="math-container">$&lt;$</span> -500</p>
  <p>2). -500 <span class="math-container">$\le$</span> x <span class="math-container">$&lt;$</span> -100</p>
  <p>3). -100 <span class="math-container">$\le$</span> x <span class="math-container">$&lt;$</span> -50</p>
  <p>4). -50 <span class="math-container">$\le$</span> x <span class="math-container">$&lt;$</span> 0</p>
  <p>5). 0 <span class="math-container">$\le$</span> x <span class="math-container">$&lt;$</span> 50</p>
  <p>6). 50 <span class="math-container">$\le$</span> x <span class="math-container">$&lt;$</span> 100</p>
  <p>7). 100 <span class="math-container">$\le$</span> x <span class="math-container">$&lt;$</span> 500</p>
  <p>8). 500 <span class="math-container">$\le$</span> x <span class="math-container">$&lt;$</span> 1000</p>
  <p>9). x &gt; 1000</p>
  <p>Therefore in this example scenario there are 9 buckets. Hence, the observations can be in range [0, 9] discretely.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-19T10:22:12.727Z
Answers.LastActivityDate: 2020-07-19T10:22:12.727Z
Answers.UserDisplayName: rert588
Body: <p>I am learning with the OpenAI gym's cart pole environment.</p>
  <p>I want to make the observation states discrete (with small stepsize) and for that purpose, I need to change two of the observations from [<span class="math-container">$
  -\infty, \infty$</span>] to some finite upper and lower limits. (By the way, these states are velocity and pole velocity at the tip).</p>
  <p>How can I change these limits in the actual gym's environment?
  Any other suggestions are also welcome.</p>
  
Tags: <reinforcement-learning><q-learning><open-ai><gym>
Title: How can I change observation states' values in OpenAI gym's cartpole environment?
UserDisplayName: SJa
ThreadText: # How can I change observation states' values in OpenAI gym's cartpole environment?
  
  (Asked by: username_0 on 2020-07-18)
  
  I am learning with the OpenAI gym's cart pole environment.
  
  I want to make the observation states discrete (with small stepsize) and for that purpose, I need to change two of the observations from [$
  -\infty, \infty$] to some finite upper and lower limits. (By the way, these states are velocity and pole velocity at the tip).
  
  How can I change these limits in the actual gym's environment?
  Any other suggestions are also welcome.
  
  **Tags:** reinforcement-learning, q-learning, open-ai, gym
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-07-19. Score: 1
  
  I don't recommend changing the rules of the environment.
  
  What you could do:
  
  Perform a method called bucketing i.e. take a value from a continuous state space see which discrete bucket it should go into and then let your agent use the bucket number as the observation.
  
  e.g. Say I do have a continuous state space with one variable in range $[-\infty,\infty]$
  
  The buckets can be as follows:
  
  0). x < -1000
  
  1). -1000 $\le$ x $<$ -500
  
  2). -500 $\le$ x $<$ -100
  
  3). -100 $\le$ x $<$ -50
  
  4). -50 $\le$ x $<$ 0
  
  5). 0 $\le$ x $<$ 50
  
  6). 50 $\le$ x $<$ 100
  
  7). 100 $\le$ x $<$ 500
  
  8). 500 $\le$ x $<$ 1000
  
  9). x > 1000
  
  Therefore in this example scenario there are 9 buckets. Hence, the observations can be in range [0, 9] discretely.
  
  
  ---
  
  