Answers.Body: <p>As the accepted answer states, the return at the current timestep is equal to the sum of discounted rewards from all future timesteps until the end of the episode. In Chapter 5 of Sutton and Barto, returns must be used to estimate the state-value and action-value functions because episode lengths are unrestricted and may be <em>greater than one</em>. In contrast, Chapter 2 deals with the very special case of multi-armed bandits in which episode lengths are always <em>equal to one</em>: The agent begins each episode in a fixed start state, takes an action, receives a reward, and then the episode terminates and the agent begins the next episode at the same start state. Therefore, <strong>a return is equivalent to a reward in Chapter 2 because all episodes have length one</strong>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-05T17:34:33.320Z
Answers.LastActivityDate: 2020-06-05T17:34:33.320Z
Answers.UserDisplayName: DeepQZero
Answers.Body: <p><strong>Return</strong> refers to the <strong>total discounted reward</strong>, starting from the current timestep.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-06-04T04:22:51.343Z
Answers.LastActivityDate: 2020-06-04T04:22:51.343Z
Answers.UserDisplayName: stoic-santiago
Body: <p>I am reading Sutton and Barto's book on reinforcement learning. I thought that reward and return were the same things. </p>
  
  <p>However, in Section 5.6 of the book, 3rd line, first paragraph, it is written:</p>
  
  <blockquote>
    <p>Whereas in Chapter 2 we averaged rewards, in Monte Carlo methods we average returns.</p>
  </blockquote>
  
  <p>What does it mean? Are rewards and returns different things?</p>
  
Tags: <reinforcement-learning><comparison><rewards><return>
Title: Is there any difference between reward and return in reinforcement learning?
UserDisplayName: SJa
ThreadText: # Is there any difference between reward and return in reinforcement learning?
  
  (Asked by: username_0 on 2020-06-04)
  
  I am reading Sutton and Barto's book on reinforcement learning. I thought that reward and return were the same things.
  
  However, in Section 5.6 of the book, 3rd line, first paragraph, it is written:
  
  > Whereas in Chapter 2 we averaged rewards, in Monte Carlo methods we average returns.
  
  What does it mean? Are rewards and returns different things?
  
  **Tags:** reinforcement-learning, comparison, rewards, return
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2020-06-04. Score: 6 | Accepted answer
  
  **Return** refers to the **total discounted reward**, starting from the current timestep.
  
  
  ---
  
  ### Answer by username_1 on 2020-06-05. Score: 3
  
  As the accepted answer states, the return at the current timestep is equal to the sum of discounted rewards from all future timesteps until the end of the episode. In Chapter 5 of Sutton and Barto, returns must be used to estimate the state-value and action-value functions because episode lengths are unrestricted and may be *greater than one*. In contrast, Chapter 2 deals with the very special case of multi-armed bandits in which episode lengths are always *equal to one*: The agent begins each episode in a fixed start state, takes an action, receives a reward, and then the episode terminates and the agent begins the next episode at the same start state. Therefore, **a return is equivalent to a reward in Chapter 2 because all episodes have length one**.
  
  
  ---
  
  