Answers.Body: <p>Because the value of the terminal state is 0 by definition. There is no further reward to be obtained once you reach the terminal state.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-08-26T18:37:20.077Z
Answers.LastActivityDate: 2021-08-26T18:37:20.077Z
Answers.UserDisplayName: tnfru
Body: <p>In the algorithm below, when <span class="math-container">$\tau + n \geq T$</span>, shouldn't the algorithm bootstrap with the value of the next state? For instance, when <span class="math-container">$T=5, \tau=3, \&amp; \; n=2$</span>, we don't bootstrap the sample return with <span class="math-container">$V_{(\tau+n)}$</span>, i.e., <span class="math-container">$V_5$</span> or the terminal state.</p>
  <p><img src="https://miro.medium.com/max/875/1*8omwhV8YaJ6jHnvOQvEMmw.png" alt="Text" /></p>
  <p>Also, on line 4, what do we mean by &quot;<strong>can take their index mod <span class="math-container">$n + 1$</span></strong>&quot;?</p>
  
Tags: <reinforcement-learning><temporal-difference-methods><model-free-methods><pseudocode>
Title: Why don't we bootstrap terminal state in n-step temporal difference prediction update equation?
UserDisplayName: user529295
ThreadText: # Why don't we bootstrap terminal state in n-step temporal difference prediction update equation?
  
  (Asked by: username_0 on 2021-08-18)
  
  In the algorithm below, when $\tau + n \geq T$, shouldn't the algorithm bootstrap with the value of the next state? For instance, when $T=5, \tau=3, \& \; n=2$, we don't bootstrap the sample return with $V\_{(\tau+n)}$, i.e., $V\_5$ or the terminal state.
  
  ![Text](https://miro.medium.com/max/875/1*8omwhV8YaJ6jHnvOQvEMmw.png)
  
  Also, on line 4, what do we mean by "**can take their index mod $n + 1$**"?
  
  **Tags:** reinforcement-learning, temporal-difference-methods, model-free-methods, pseudocode
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-08-26. Score: 1
  
  Because the value of the terminal state is 0 by definition. There is no further reward to be obtained once you reach the terminal state.
  
  
  ---
  
  