Answers.Body: <p>All right, I figured it out. trajectories need not have the same starting state because the distribution of <span class="math-container">$s_0$</span> is drawn from a distribution D (mentioned in the paper). Had been confused because many of the code implementations on github focus on trajectories starting from the same state. </p>
  
  <p>Hope this helps everyone !</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-03T13:40:55.220Z
Answers.Text: Maybe you should say where/how you discovered that $s_0$ is drawn from $D$. Does the paper you mention in your question state this?
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-03T13:54:02.683Z
Answers.Text: Yes the paper mentioned above states this. The confusion came when I was reading a blog post about IRL. I might have misunderstood when the blog said that "The Starting position(state) of the bot is fixed, as according to the IRL algorithm it is necessary that the starting state is same for all the iterations.", from https://jangirrishabh.github.io/2016/07/09/virtual-car-IRL/ leading to some doubts that I had
Answers.UserDisplayName: calveeen
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-04-03T13:10:42.477Z
Answers.LastActivityDate: 2020-04-03T13:54:36.873Z
Answers.LastEditDate: 2020-04-03T13:54:36.873Z
Answers.UserDisplayName: calveeen
Body: <p>In the apprenticeship learning algorithm described by Ng et al. in <a href="https://ai.stanford.edu/~ang/papers/icml04-apprentice.pdf" rel="nofollow noreferrer">Apprenticeship Learning via Inverse Reinforcement Learning</a>, they mention that expert trajectories come in the form of <span class="math-container">$\{s_0^i, s_1^i\, ...\}_{i=1}^m$</span>. However, they also mentioned that <span class="math-container">$s_0 $</span> is drawn from distribution D. Do all expert trajectories then have to have the same starting state? Why is it not possible to compute the feature expectation based on a single trajectory?</p>
  
Tags: <reinforcement-learning><papers><rewards><apprenticeship-learning>
Title: Do all expert trajectories have the same starting state in apprenticeship learning?
UserDisplayName: calveeen
ThreadText: # Do all expert trajectories have the same starting state in apprenticeship learning?
  
  (Asked by: username_0 on 2020-03-27)
  
  In the apprenticeship learning algorithm described by Ng et al. in [Apprenticeship Learning via Inverse Reinforcement Learning](https://ai.stanford.edu/~ang/papers/icml04-apprentice.pdf), they mention that expert trajectories come in the form of $\{s\_0^i, s\_1^i\, ...\}\_{i=1}^m$. However, they also mentioned that $s\_0 $ is drawn from distribution D. Do all expert trajectories then have to have the same starting state? Why is it not possible to compute the feature expectation based on a single trajectory?
  
  **Tags:** reinforcement-learning, papers, rewards, apprenticeship-learning
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2020-04-03. Score: 3 | Accepted answer
  
  All right, I figured it out. trajectories need not have the same starting state because the distribution of $s\_0$ is drawn from a distribution D (mentioned in the paper). Had been confused because many of the code implementations on github focus on trajectories starting from the same state.
  
  Hope this helps everyone !
  
  > **Comments:**
  > **username_1**: Maybe you should say where/how you discovered that $s_0$ is drawn from $D$. Does the paper you mention in your question state this?  
  > **username_0**: Yes the paper mentioned above states this. The confusion came when I was reading a blog post about IRL. I might have misunderstood when the blog said that "The Starting position(state) of the bot is fixed, as according to the IRL algorithm it is necessary that the starting state is same for all the iterations.", from https://jangirrishabh.github.io/2016/07/09/virtual-car-IRL/ leading to some doubts that I had  
  
  ---
  
  