Answers.Body: <p><span class="math-container">$ R + \gamma {max}_aQ(S^{\prime},a) $</span> is the target Q value for <span class="math-container">$(S,A)$</span>.  The present estimate <span class="math-container">$Q(S,A)$</span> is subtracted from it to arrive at the error.  The absolute value is taken to arrive at the absolute error.  This absolute error is used to prioritize the next state to update.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-05-25T03:29:02.273Z
Answers.LastActivityDate: 2024-05-25T03:29:02.273Z
Answers.UserDisplayName: foreverska
Body: <p>Could someone explain the red line (especially, the meaning of the difference) in prioritized sweeping algorithm below?</p>
  <p>Sutton-Barto, page 170:</p>
  <p><a href="https://i.sstatic.net/WxZjHPDw.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/WxZjHPDw.png" alt="enter image description here" /></a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-05-25T07:16:29.423Z
Comments.Text: Why are you so persistent in using screen shots of text, with no readable summary? You know better by now. Please stop doing this.
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-05-25T16:54:22.780Z
Comments.Text: I will try your follow your advice from now on unless the writing will take too much time.
Comments.UserDisplayName: DSPinfinity
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-05-26T06:58:44.780Z
Comments.Text: Thank you. You don't have to replicate an entire screenshot, but you should always reproduce the relevant parts to the question in text (i.e. don't underline text in an image), and ideally enough of the rest to give a little context. Here, copying line (e) as text instead of underlining it in an image would probably be enough.
Comments.UserDisplayName: Neil Slater
Tags: <reinforcement-learning><sutton-barto><prioritized-sweeping>
Title: Unclear line in prioritized sweeping algorithm
UserDisplayName: DSPinfinity
ThreadText: # Unclear line in prioritized sweeping algorithm
  
  (Asked by: username_0 on 2024-05-24)
  
  Could someone explain the red line (especially, the meaning of the difference) in prioritized sweeping algorithm below?
  
  Sutton-Barto, page 170:
  
  [![enter image description here](https://i.sstatic.net/WxZjHPDw.png)](https://i.sstatic.net/WxZjHPDw.png)
  
  **Tags:** reinforcement-learning, sutton-barto, prioritized-sweeping
  > **Comments:**
  > **username_1**: Why are you so persistent in using screen shots of text, with no readable summary? You know better by now. Please stop doing this.  
  > **username_0**: I will try your follow your advice from now on unless the writing will take too much time.  
  > **username_1**: Thank you. You don't have to replicate an entire screenshot, but you should always reproduce the relevant parts to the question in text (i.e. don't underline text in an image), and ideally enough of the rest to give a little context. Here, copying line (e) as text instead of underlining it in an image would probably be enough.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2024-05-25. Score: 2 | Accepted answer
  
  $ R + \gamma {max}\_aQ(S^{\prime},a) $ is the target Q value for $(S,A)$. The present estimate $Q(S,A)$ is subtracted from it to arrive at the error. The absolute value is taken to arrive at the absolute error. This absolute error is used to prioritize the next state to update.
  
  
  ---
  
  