Answers.Body: <p>To answer your question, the specifics of some of the OpenAI Gym environments can be found on their <a href="https://github.com/openai/gym/wiki/MountainCar-v0" rel="nofollow noreferrer">wiki</a>:</p>
  <blockquote>
  <p>The episode ends when you reach 0.5 position, or if 200 iterations are reached.</p>
  </blockquote>
  <p>There is a deeper question in what you asked, though:</p>
  <blockquote>
  <p>My initial understanding was that an episode should end when the Car reaches the flagpost.</p>
  </blockquote>
  <p>The environment certainly could be set up that way. Limiting the number of steps per episode has the immediate benefit of forcing the agent to reach the goal state in a fixed amount of time, which often results in a speedier trajectory by the agent (MountainCar-v0 further penalizes long trajectories through the reward signal). Also, the underlying learning algorithm may only perform policy updates after completion of the episode. If the agent will never reach the goal state under its current policy (i.e. the policy is very bad, lacks much randomness, etc.), then terminating the episode after a fixed amount of time will ensure that the agent is able to perform a policy update and try a new policy on the next episode (alternatively, the learning algorithm could perform a policy update during the episode). There is a branch of tasks called <em>continuing tasks</em> which never terminate (see Section 3.3 of <a href="http://www.incompleteideas.net/book/the-book-2nd.html" rel="nofollow noreferrer">Sutton and Barto</a>), so the choice to limit the number of steps per episode is heavily dependent on the task at hand and the choice of learning algorithm.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-17T15:45:40.113Z
Answers.LastActivityDate: 2020-07-17T15:45:40.113Z
Answers.UserDisplayName: DeepQZero
Answers.Body: <p>The episode ends when either the car reaches the goal, or a maximum number of timesteps has passed. By default the episode will terminate after 200 steps. You can customize this with the <code>_max_episode_steps</code> attribute of the environment.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-17T15:08:00.153Z
Answers.LastActivityDate: 2020-07-17T15:08:00.153Z
Answers.UserDisplayName: harwiltz
Body: <p>I am working on OpenAI's &quot;MountainCar-v0&quot; environment.  In this environment, each step that an agent takes returns (among other values) the variable named <code>done</code> of type boolean. The variable gets a <code>True</code> value when the episode ends. However, I am not sure how each episode ends. My initial understanding was that an episode should end when the car reaches the flagpost. However, that is not the case.</p>
  <p>What are the states/actions under which the episode terminates in this environment?</p>
  
Tags: <reinforcement-learning><environment><gym>
Title: How does an episode end in OpenAI Gym's "MountainCar-v0" environment?
UserDisplayName: SJa
ThreadText: # How does an episode end in OpenAI Gym's "MountainCar-v0" environment?
  
  (Asked by: username_0 on 2020-07-17)
  
  I am working on OpenAI's "MountainCar-v0" environment. In this environment, each step that an agent takes returns (among other values) the variable named `done` of type boolean. The variable gets a `True` value when the episode ends. However, I am not sure how each episode ends. My initial understanding was that an episode should end when the car reaches the flagpost. However, that is not the case.
  
  What are the states/actions under which the episode terminates in this environment?
  
  **Tags:** reinforcement-learning, environment, gym
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2020-07-17. Score: 3 | Accepted answer
  
  The episode ends when either the car reaches the goal, or a maximum number of timesteps has passed. By default the episode will terminate after 200 steps. You can customize this with the `_max_episode_steps` attribute of the environment.
  
  
  ---
  
  ### Answer by username_1 on 2020-07-17. Score: 3
  
  To answer your question, the specifics of some of the OpenAI Gym environments can be found on their [wiki](https://github.com/openai/gym/wiki/MountainCar-v0):
  
  > The episode ends when you reach 0.5 position, or if 200 iterations are reached.
  
  There is a deeper question in what you asked, though:
  
  > My initial understanding was that an episode should end when the Car reaches the flagpost.
  
  The environment certainly could be set up that way. Limiting the number of steps per episode has the immediate benefit of forcing the agent to reach the goal state in a fixed amount of time, which often results in a speedier trajectory by the agent (MountainCar-v0 further penalizes long trajectories through the reward signal). Also, the underlying learning algorithm may only perform policy updates after completion of the episode. If the agent will never reach the goal state under its current policy (i.e. the policy is very bad, lacks much randomness, etc.), then terminating the episode after a fixed amount of time will ensure that the agent is able to perform a policy update and try a new policy on the next episode (alternatively, the learning algorithm could perform a policy update during the episode). There is a branch of tasks called *continuing tasks* which never terminate (see Section 3.3 of [Sutton and Barto](http://www.incompleteideas.net/book/the-book-2nd.html)), so the choice to limit the number of steps per episode is heavily dependent on the task at hand and the choice of learning algorithm.
  
  
  ---
  
  