Answers.Body: <p>The authors of the paper <a href="https://arxiv.org/pdf/1710.11248.pdf" rel="nofollow noreferrer">Learning Robust Rewards with Adversarial Inverse Reinforcement Learning</a> (2018, published in ICRL), which introduced the inverse RL technique <strong>AIRL</strong>, argue that <strong>GAIL</strong> fails to generalize to different environment's dynamics. Specifically, in section 7.2 (p. 7), they describe an experiment where they disable and shrink the two front legs of the <a href="https://gym.openai.com/envs/Ant-v2/" rel="nofollow noreferrer">ant</a>, then, based on the results, they conclude</p>
  <blockquote>
  <p>GAIL learns successfully in the training domain, but does not acquire a representation that is suitable for transfer to test domains.</p>
  </blockquote>
  <p>On the other hand, according to their experiments, AIRL is more robust to changes in the environment's dynamics.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-11-05T23:40:15.013Z
Answers.LastActivityDate: 2020-11-05T23:45:50.413Z
Answers.LastEditDate: 2020-11-05T23:45:50.413Z
Answers.UserDisplayName: nbro
Body: <p>Is the <a href="https://papers.nips.cc/paper/2016/file/cc7e2b878868cbae992d1fb743995d8f-Paper.pdf" rel="nofollow noreferrer">GAIL</a> applicable if the expert's trajectories (sample data) are for the same task but are in a different environment (modified but will not be completely different)?</p>
  <p>My gut feeling is, yes, otherwise we can just simply adopt behavioural cloning. Furthermore, since the expert's trajectories are from a different environment, the dimension/length of state-action pairs will most likely be different. Will those trajectories still be useful for GAIL training?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-11-05T23:23:14.823Z
Comments.Text: What do you mean by "behavioural cloning" here?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-11-06T11:46:27.293Z
Comments.Text: I mean the supervised imitation learning approach where the agent tries to copy the demonstrated labeled data.
Comments.UserDisplayName: Sam
Tags: <reinforcement-learning><generative-adversarial-networks><generative-model><imitation-learning><gail>
Title: Is GAIL applicable if the expert's trajectories are for the same task but are in a different environment?
UserDisplayName: Sam
ThreadText: # Is GAIL applicable if the expert's trajectories are for the same task but are in a different environment?
  
  (Asked by: username_0 on 2020-06-30)
  
  Is the [GAIL](https://papers.nips.cc/paper/2016/file/cc7e2b878868cbae992d1fb743995d8f-Paper.pdf) applicable if the expert's trajectories (sample data) are for the same task but are in a different environment (modified but will not be completely different)?
  
  My gut feeling is, yes, otherwise we can just simply adopt behavioural cloning. Furthermore, since the expert's trajectories are from a different environment, the dimension/length of state-action pairs will most likely be different. Will those trajectories still be useful for GAIL training?
  
  **Tags:** reinforcement-learning, generative-adversarial-networks, generative-model, imitation-learning, gail
  
  > **Comments:**
  > **username_1**: What do you mean by "behavioural cloning" here?  
  > **username_0**: I mean the supervised imitation learning approach where the agent tries to copy the demonstrated labeled data.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-11-05. Score: 1 | Accepted answer
  
  The authors of the paper [Learning Robust Rewards with Adversarial Inverse Reinforcement Learning](https://arxiv.org/pdf/1710.11248.pdf) (2018, published in ICRL), which introduced the inverse RL technique **AIRL**, argue that **GAIL** fails to generalize to different environment's dynamics. Specifically, in section 7.2 (p. 7), they describe an experiment where they disable and shrink the two front legs of the [ant](https://gym.openai.com/envs/Ant-v2/), then, based on the results, they conclude
  
  > GAIL learns successfully in the training domain, but does not acquire a representation that is suitable for transfer to test domains.
  
  On the other hand, according to their experiments, AIRL is more robust to changes in the environment's dynamics.
  
  
  ---
  
  