Answers.Body: <p>I found out that</p>
  <p><span class="math-container">$\sum_{r\in R} r p(r|s,a,s') = \sum_{r\in R} r \frac{p(s,a,s',r)}{p(s,a,s')} = \sum_{r\in R} r \frac{p(s',r|s,a)}{p(s'|s,a)} $</span> by cancelling <span class="math-container">$p(s,a)$</span>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-06-13T10:30:40.833Z
Answers.LastActivityDate: 2023-06-13T10:30:40.833Z
Answers.UserDisplayName: tesio
Body: <p><a href="https://i.sstatic.net/IGMRZ.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/IGMRZ.png" alt="enter image description here" /></a></p>
  <p>I don't understand the last equality.</p>
  <p>Here is my derivation
  <span class="math-container">$r(s,a,s')=\sum_{r\in R} r p(r|s,a,s')=\sum_{r\in R} r \frac{p(s,a,s'|r)p(r)}{p(s,a,s')}$</span></p>
  <p>Could you give me the correct steps to derive them?</p>
  <p>Thank you</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-06-14T11:11:32.647Z
Comments.Text: Possibly a duplicate of [Why does the definition of the reward function $r(s, a, s')$ involve the term $p(s' \mid s, a)$?](https://ai.stackexchange.com/questions/20244/why-does-the-definition-of-the-reward-function-rs-a-s-involve-the-term-p), although maybe the answers don't provide the full derivation.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-06-14T11:12:49.970Z
Comments.Text: There's even another question which may be a duplicate of this. I think Dennis provided an answer to it.
Comments.UserDisplayName: nbro
Tags: <reinforcement-learning><reward-functions>
Title: Could you explain the derivation of the expectation equation of equation 3.6 in Sutton & Barto?
UserDisplayName: tesio
ThreadText: # Could you explain the derivation of the expectation equation of equation 3.6 in Sutton & Barto?
  
  (Asked by: username_0 on 2023-06-13)
  
  [![enter image description here](https://i.sstatic.net/IGMRZ.png)](https://i.sstatic.net/IGMRZ.png)
  
  I don't understand the last equality.
  
  Here is my derivation
  $r(s,a,s')=\sum\_{r\in R} r p(r|s,a,s')=\sum\_{r\in R} r \frac{p(s,a,s'|r)p(r)}{p(s,a,s')}$
  
  Could you give me the correct steps to derive them?
  
  Thank you
  
  **Tags:** reinforcement-learning, reward-functions
  
  > **Comments:**
  > **username_1**: Possibly a duplicate of [Why does the definition of the reward function $r(s, a, s')$ involve the term $p(s' \mid s, a)$?](https://ai.stackexchange.com/questions/20244/why-does-the-definition-of-the-reward-function-rs-a-s-involve-the-term-p), although maybe the answers don't provide the full derivation.  
  > **username_1**: There's even another question which may be a duplicate of this. I think Dennis provided an answer to it.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2023-06-13. Score: 0
  
  I found out that
  
  $\sum\_{r\in R} r p(r|s,a,s') = \sum\_{r\in R} r \frac{p(s,a,s',r)}{p(s,a,s')} = \sum\_{r\in R} r \frac{p(s',r|s,a)}{p(s'|s,a)} $ by cancelling $p(s,a)$.
  
  
  ---
  
  