Answers.Body: <blockquote>
    <p>Can an RL algorithm trained in one environment be succesfull on a different one? </p>
  </blockquote>
  
  <p>Strictly the answer here is "no". You train an agent to solve a single environment. If a second environment is similar enough, you can probably re-train the agent on the new environment with little or no changes to the agent, and to get results just as good in the second agent. </p>
  
  <p>In some cases, you could even start with the trained agent and "fine tune" it to the new environment. That probably would not work well with the labyrinth example though, especially if the start and end points are moved.</p>
  
  <blockquote>
    <p>Example: If I train a model to go through one labyrinth, could this model also go through different but similar labyrinth or would it need a new training process?</p>
  </blockquote>
  
  <p>This could be subtly different. It is down to how you define the environment. It is possible to define an environment not as "this maze" but for example "all possible mazes in a 30x20 grid pattern". To do so, you need to make the maze configuration part of the environment's <em>state</em>. </p>
  
  <p>Expanding from a singular environment example to all similar environments has a cost. In the labyrinth example, this is significant. A single 30x20 maze has 600 states, which is a trivial problem in RL. All possible mazes of the same size has closer to <span class="math-container">$2^{1200}$</span> states, which is much larger and requires different kinds of approaches, different RL methods.</p>
  
  <p>You would expect to train on many example mazes (typically a randomly generated new one for each episode) and use some kind of generalisation - e.g. a neural network, maybe a CNN due to the grid design - when handling the value function or policy function. Training time for a single maze will be under a second, and the policy will be perfect. Training time for all possible mazes measured in hours or days and the policy will still get things wrong from time-to-time.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-09-14T09:35:16.490Z
Answers.Text: Labyrinth was just the easiest example I could think of, but I see the point now :)
Answers.UserDisplayName: GKozinski
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-09-13T10:42:58.610Z
Answers.LastActivityDate: 2019-09-13T10:42:58.610Z
Answers.UserDisplayName: Neil Slater
Body: <p><em>Can an RL algorithm trained in one environment be successful in a different one?</em></p>
  
  <p>For example, if I train a model to go through one labyrinth, could this model also go through a different but similar labyrinth or would it need a new training process?</p>
  
  <p>By similar, I mean like these two:</p>
  
  <p><a href="https://i.sstatic.net/hVGOA.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/hVGOA.jpg" alt="enter image description here"></a></p>
  
  <p><a href="https://i.sstatic.net/jnEKx.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/jnEKx.jpg" alt="enter image description here"></a></p>
  
  <p>But with this one being not similar:</p>
  
  <p><a href="https://i.sstatic.net/4koPq.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/4koPq.jpg" alt="enter image description here"></a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-09-13T10:19:46.390Z
Comments.Text: I used labyrinth only as as an example to more general aspect, but yeah, I edited the question to explain a bit more.
Comments.UserDisplayName: GKozinski
Tags: <reinforcement-learning><environment>
Title: Can an RL algorithm trained in one environment be successful in a different one?
UserDisplayName: GKozinski
ThreadText: # Can an RL algorithm trained in one environment be successful in a different one?
  
  (Asked by: username_0 on 2019-09-13)
  
  *Can an RL algorithm trained in one environment be successful in a different one?*
  
  For example, if I train a model to go through one labyrinth, could this model also go through a different but similar labyrinth or would it need a new training process?
  
  By similar, I mean like these two:
  
  [![enter image description here](https://i.sstatic.net/hVGOA.jpg)](https://i.sstatic.net/hVGOA.jpg)
  
  [![enter image description here](https://i.sstatic.net/jnEKx.jpg)](https://i.sstatic.net/jnEKx.jpg)
  
  But with this one being not similar:
  
  [![enter image description here](https://i.sstatic.net/4koPq.jpg)](https://i.sstatic.net/4koPq.jpg)
  
  **Tags:** reinforcement-learning, environment
  
  > **Comments:**
  > **username_0**: I used labyrinth only as as an example to more general aspect, but yeah, I edited the question to explain a bit more.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-09-13. Score: 0 | Accepted answer
  
  > Can an RL algorithm trained in one environment be succesfull on a different one?
  
  Strictly the answer here is "no". You train an agent to solve a single environment. If a second environment is similar enough, you can probably re-train the agent on the new environment with little or no changes to the agent, and to get results just as good in the second agent.
  
  In some cases, you could even start with the trained agent and "fine tune" it to the new environment. That probably would not work well with the labyrinth example though, especially if the start and end points are moved.
  
  > Example: If I train a model to go through one labyrinth, could this model also go through different but similar labyrinth or would it need a new training process?
  
  This could be subtly different. It is down to how you define the environment. It is possible to define an environment not as "this maze" but for example "all possible mazes in a 30x20 grid pattern". To do so, you need to make the maze configuration part of the environment's *state*.
  
  Expanding from a singular environment example to all similar environments has a cost. In the labyrinth example, this is significant. A single 30x20 maze has 600 states, which is a trivial problem in RL. All possible mazes of the same size has closer to $2^{1200}$ states, which is much larger and requires different kinds of approaches, different RL methods.
  
  You would expect to train on many example mazes (typically a randomly generated new one for each episode) and use some kind of generalisation - e.g. a neural network, maybe a CNN due to the grid design - when handling the value function or policy function. Training time for a single maze will be under a second, and the policy will be perfect. Training time for all possible mazes measured in hours or days and the policy will still get things wrong from time-to-time.
  
  > **Comments:**
  > **username_0**: Labyrinth was just the easiest example I could think of, but I see the point now :)  
  
  ---
  
  