<p>I found out that</p>
<p><span class="math-container">$\sum_{r\in R} r p(r|s,a,s') = \sum_{r\in R} r \frac{p(s,a,s',r)}{p(s,a,s')} = \sum_{r\in R} r \frac{p(s',r|s,a)}{p(s'|s,a)} $</span> by cancelling <span class="math-container">$p(s,a)$</span>.</p>

CC BY-SA 4.0
2023-06-13T10:30:40.833Z
2023-06-13T10:30:40.833Z
tesio
<p><a href="https://i.sstatic.net/IGMRZ.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/IGMRZ.png" alt="enter image description here" /></a></p>
<p>I don't understand the last equality.</p>
<p>Here is my derivation
<span class="math-container">$r(s,a,s')=\sum_{r\in R} r p(r|s,a,s')=\sum_{r\in R} r \frac{p(s,a,s'|r)p(r)}{p(s,a,s')}$</span></p>
<p>Could you give me the correct steps to derive them?</p>
<p>Thank you</p>

CC BY-SA 4.0
2023-06-14T11:11:32.647Z
Possibly a duplicate of [Why does the definition of the reward function $r(s, a, s')$ involve the term $p(s' \mid s, a)$?](https://ai.stackexchange.com/questions/20244/why-does-the-definition-of-the-reward-function-rs-a-s-involve-the-term-p), although maybe the answers don't provide the full derivation.
nbro
CC BY-SA 4.0
2023-06-14T11:12:49.970Z
There's even another question which may be a duplicate of this. I think Dennis provided an answer to it.
nbro
<reinforcement-learning><reward-functions>
Could you explain the derivation of the expectation equation of equation 3.6 in Sutton & Barto?
tesio
# Could you explain the derivation of the expectation equation of equation 3.6 in Sutton & Barto?

(Asked by: username_0 on 2023-06-13)

[![enter image description here](https://i.sstatic.net/IGMRZ.png)](https://i.sstatic.net/IGMRZ.png)

I don't understand the last equality.

Here is my derivation
$r(s,a,s')=\sum\_{r\in R} r p(r|s,a,s')=\sum\_{r\in R} r \frac{p(s,a,s'|r)p(r)}{p(s,a,s')}$

Could you give me the correct steps to derive them?

Thank you

**Tags:** reinforcement-learning, reward-functions

> **Comments:**
> **username_1**: Possibly a duplicate of [Why does the definition of the reward function $r(s, a, s')$ involve the term $p(s' \mid s, a)$?](https://ai.stackexchange.com/questions/20244/why-does-the-definition-of-the-reward-function-rs-a-s-involve-the-term-p), although maybe the answers don't provide the full derivation.  
> **username_1**: There's even another question which may be a duplicate of this. I think Dennis provided an answer to it.  

---

## 1 answers

### Answer by username_0 on 2023-06-13. Score: 0

I found out that

$\sum\_{r\in R} r p(r|s,a,s') = \sum\_{r\in R} r \frac{p(s,a,s',r)}{p(s,a,s')} = \sum\_{r\in R} r \frac{p(s',r|s,a)}{p(s'|s,a)} $ by cancelling $p(s,a)$.


---

