Answers.Body: <p>Sutton and Barto explain it themselves <a href="http://incompleteideas.net/book/RLbook2020.pdf#page=136" rel="nofollow noreferrer">in section 5.9</a>. I post it with a bit of context. <strong>The equation you're looking for is 5.13</strong>.<a href="https://i.sstatic.net/3TSr8.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/3TSr8.png" alt="Excerpt from Section 5.9" /></a></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-08-31T18:05:23.123Z
Answers.LastActivityDate: 2021-08-31T18:05:23.123Z
Answers.UserDisplayName: tnfru
Body: <p>In Sutton &amp; Barto (2nd edition), the following is mentioned <a href="http://incompleteideas.net/book/RLbook2020.pdf#page=172" rel="nofollow noreferrer">on page 150 (p. 172 of the pdf), section 7.4</a>:</p>
  <blockquote>
  <p>the importance sampling ratio has expected value one (Section 5.9) and is uncorrelated with the estimate.</p>
  </blockquote>
  <p>How can we prove the importance sampling ratio is uncorrelated with the estimate?</p>
  
Tags: <proofs><off-policy-methods><sutton-barto><importance-sampling><model-free-methods>
Title: How to prove importance sampling ratio is uncorrelated with action-value (or state-value) estimate?
UserDisplayName: user529295
ThreadText: # How to prove importance sampling ratio is uncorrelated with action-value (or state-value) estimate?
  
  (Asked by: username_0 on 2021-08-20)
  
  In Sutton & Barto (2nd edition), the following is mentioned [on page 150 (p. 172 of the pdf), section 7.4](http://incompleteideas.net/book/RLbook2020.pdf#page=172):
  
  > the importance sampling ratio has expected value one (Section 5.9) and is uncorrelated with the estimate.
  
  How can we prove the importance sampling ratio is uncorrelated with the estimate?
  
  **Tags:** proofs, off-policy-methods, sutton-barto, importance-sampling, model-free-methods
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-08-31. Score: 1
  
  Sutton and Barto explain it themselves [in section 5.9](http://incompleteideas.net/book/RLbook2020.pdf#page=136). I post it with a bit of context. **The equation you're looking for is 5.13**.[![Excerpt from Section 5.9](https://i.sstatic.net/3TSr8.png)](https://i.sstatic.net/3TSr8.png)
  
  
  ---
  
  