Answers.Body: <p>A Bellman backup is an application of a Bellman operator. For example, the step</p>
  <p><span class="math-container">$$
  V(x)\leftarrow \alpha(R + \mathbf{E}[V(x')]) + (1-\alpha)V(x)
  $$</span></p>
  <p>Is a Bellman backup for some learning rate <span class="math-container">$\alpha$</span>.</p>
  <p>A Bellman error is</p>
  <p><span class="math-container">$$
  d(V(x), R + \mathbf{E}[V(x')])
  $$</span></p>
  <p>for some metric <span class="math-container">$d$</span>, usually <span class="math-container">$d(x, y) = (x-y)^2$</span>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-06-28T12:55:30.817Z
Answers.Text: What does 'backup' refer to here?
Answers.UserDisplayName: user529295
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-06-28T13:02:48.343Z
Answers.Text: It refers to propagating information from later states to earlier ones (backward in time sorta)
Answers.UserDisplayName: harwiltz
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-06-28T13:12:04.727Z
Answers.Text: It may be a good idea to 1. also provide the [figures of the backups e.g. that you can find in Sutton and Barto's book](http://incompleteideas.net/book/first/figures/figures.html), 2. to link the OP to [this question](https://ai.stackexchange.com/q/11057/2444) about what the Bellman operator is and 3. explain the symbols in your answer.
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-06-28T12:01:34.163Z
Answers.LastActivityDate: 2021-06-28T12:01:34.163Z
Answers.UserDisplayName: harwiltz
Body: <p>Some RL literature use terms such as: 'Bellman backup' and 'Bellman error'. What do these terms refer to?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-06-28T13:15:20.210Z
Comments.Text: There's already an answer that addresses both concerns/questions, but, please, next time, focus on one question per post, although, in this case, the terms are highly related (but I still think these "simple" questions could have been asked in separate posts). It may also be a good idea to provide more context (e.g. a link to an article that mentions these terms), although, again, in this case, anyone familiar with RL would be able to understand the question.
Comments.UserDisplayName: nbro
Tags: <reinforcement-learning><terminology><bellman-equations><bellman-operators>
Title: What do the terms 'Bellman backup' and 'Bellman error' mean?
UserDisplayName: user529295
ThreadText: # What do the terms 'Bellman backup' and 'Bellman error' mean?
  
  (Asked by: username_0 on 2021-06-28)
  
  Some RL literature use terms such as: 'Bellman backup' and 'Bellman error'. What do these terms refer to?
  
  **Tags:** reinforcement-learning, terminology, bellman-equations, bellman-operators
  
  > **Comments:**
  > **username_1**: There's already an answer that addresses both concerns/questions, but, please, next time, focus on one question per post, although, in this case, the terms are highly related (but I still think these "simple" questions could have been asked in separate posts). It may also be a good idea to provide more context (e.g. a link to an article that mentions these terms), although, again, in this case, anyone familiar with RL would be able to understand the question.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2021-06-28. Score: 3 | Accepted answer
  
  A Bellman backup is an application of a Bellman operator. For example, the step
  
  $$
  V(x)\leftarrow \alpha(R + \mathbf{E}[V(x')]) + (1-\alpha)V(x)
  $$
  
  Is a Bellman backup for some learning rate $\alpha$.
  
  A Bellman error is
  
  $$
  d(V(x), R + \mathbf{E}[V(x')])
  $$
  
  for some metric $d$, usually $d(x, y) = (x-y)^2$.
  
  > **Comments:**
  > **username_0**: What does 'backup' refer to here?  
  > **username_2**: It refers to propagating information from later states to earlier ones (backward in time sorta)  
  > **username_1**: It may be a good idea to 1. also provide the [figures of the backups e.g. that you can find in Sutton and Barto's book](http://incompleteideas.net/book/first/figures/figures.html), 2. to link the OP to [this question](https://ai.stackexchange.com/q/11057/2444) about what the Bellman operator is and 3. explain the symbols in your answer.  
  
  ---
  
  