Answers.Body: <p>The dot (<span class="math-container">$.$</span>) at the end of <span class="math-container">$T(s,a,.)$</span> shows all possible states that we can go from state <span class="math-container">$S$</span> by doing action <span class="math-container">$a$</span>. As you know there are some probabilities here for choosing those states, that the sum of these probabilities is equal to 1. Hence, <span class="math-container">$T(s,a,.)$</span> is a probability distribution.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-29T16:06:27.030Z
Answers.LastActivityDate: 2020-03-29T16:12:42.343Z
Answers.LastEditDate: 2020-03-29T16:12:42.343Z
Answers.UserDisplayName: OmG
Body: <p>I have been seeing notations on Expectations with their respective subscripts such as <span class="math-container">$E_{s_0 \sim D}[V^\pi (s_0)] = \Sigma_{t=0}^\infty[\gamma^t\phi(s_t)]$</span>. This equation is taken from <a href="https://ai.stanford.edu/~ang/papers/icml04-apprentice.pdf" rel="nofollow noreferrer">https://ai.stanford.edu/~ang/papers/icml04-apprentice.pdf</a> and <span class="math-container">$Q^\pi(s,a,R) = R(s) + \gamma E_{s'\sim T(s,a,\cdot)}[V^\pi(s',R)]$</span> ,in the case of the Bayesian IRL paper.(<a href="https://www.aaai.org/Papers/IJCAI/2007/IJCAI07-416.pdf" rel="nofollow noreferrer">https://www.aaai.org/Papers/IJCAI/2007/IJCAI07-416.pdf</a>) </p>
  
  <p>I understand that <span class="math-container">$s_0 \sim D$</span> means that the starting state <span class="math-container">$s_0$</span> is drawn from a distribution of starting states <span class="math-container">$D$</span>. But how do we understand the latter with subscript <span class="math-container">${s'\sim T(s,a,\cdot)}$</span> ? How is <span class="math-container">$s'$</span> drawn from a distribution of transition probabilities?</p>
  
Tags: <reinforcement-learning><notation><expectation><apprenticeship-learning>
Title: What does the notation ${s'\sim T(s,a,\cdot)}$ mean?
UserDisplayName: calveeen
ThreadText: # What does the notation ${s'\sim T(s,a,\cdot)}$ mean?
  
  (Asked by: username_0 on 2020-03-29)
  
  I have been seeing notations on Expectations with their respective subscripts such as $E\_{s\_0 \sim D}[V^\pi (s\_0)] = \Sigma\_{t=0}^\infty[\gamma^t\phi(s\_t)]$. This equation is taken from <https://ai.stanford.edu/~ang/papers/icml04-apprentice.pdf> and $Q^\pi(s,a,R) = R(s) + \gamma E\_{s'\sim T(s,a,\cdot)}[V^\pi(s',R)]$ ,in the case of the Bayesian IRL paper.(<https://www.aaai.org/Papers/IJCAI/2007/IJCAI07-416.pdf>)
  
  I understand that $s\_0 \sim D$ means that the starting state $s\_0$ is drawn from a distribution of starting states $D$. But how do we understand the latter with subscript ${s'\sim T(s,a,\cdot)}$ ? How is $s'$ drawn from a distribution of transition probabilities?
  
  **Tags:** reinforcement-learning, notation, expectation, apprenticeship-learning
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-03-29. Score: 2 | Accepted answer
  
  The dot ($.$) at the end of $T(s,a,.)$ shows all possible states that we can go from state $S$ by doing action $a$. As you know there are some probabilities here for choosing those states, that the sum of these probabilities is equal to 1. Hence, $T(s,a,.)$ is a probability distribution.
  
  
  ---
  
  