Answers.Body: <p>The policy is used in determining the next sequence of state-action pairs in the next episode. This means that the policy is determining indirectly the next <code>Gt</code></p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-07-01T20:16:25.230Z
Answers.LastActivityDate: 2021-07-01T20:16:25.230Z
Answers.UserDisplayName: alwayscurious
Body: <p>In his lecture 5 of the course &quot;Reinforcement Learning&quot;, David Silver introduced GLIE Monte-Carlo Control.</p>
  <p><a href="https://i.sstatic.net/JaYl7.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/JaYl7.png" alt="enter image description here" /></a></p>
  <p>I understand that we do policy evaluation for one step and then policy improvement. My question is how does the improved policy come into play in this GLIE algorithm?</p>
  <p>Is <code>Gt</code> (return) based on the policy somehow? is that where the new policy comes in?
  Asked another way, how are policy evaluation and policy improvement connected in this image?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-07-01T11:09:36.157Z
Comments.Text: Hello. Welcome to AI SE! I just want to let you know that you case use latex on this site, so you may want to format your math symbols it ;)
Comments.UserDisplayName: nbro
Tags: <reinforcement-learning><monte-carlo-methods>
Title: GLIE MC control (reinforcement learning): how the policy affects evaluation?
UserDisplayName: alwayscurious
ThreadText: # GLIE MC control (reinforcement learning): how the policy affects evaluation?
  
  (Asked by: username_0 on 2021-06-30)
  
  In his lecture 5 of the course "Reinforcement Learning", David Silver introduced GLIE Monte-Carlo Control.
  
  [![enter image description here](https://i.sstatic.net/JaYl7.png)](https://i.sstatic.net/JaYl7.png)
  
  I understand that we do policy evaluation for one step and then policy improvement. My question is how does the improved policy come into play in this GLIE algorithm?
  
  Is `Gt` (return) based on the policy somehow? is that where the new policy comes in?
  Asked another way, how are policy evaluation and policy improvement connected in this image?
  
  **Tags:** reinforcement-learning, monte-carlo-methods
  
  > **Comments:**
  > **username_1**: Hello. Welcome to AI SE! I just want to let you know that you case use latex on this site, so you may want to format your math symbols it ;)  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2021-07-01. Score: 0 | Accepted answer
  
  The policy is used in determining the next sequence of state-action pairs in the next episode. This means that the policy is determining indirectly the next `Gt`
  
  
  ---
  
  