<p>As the accepted answer states, the return at the current timestep is equal to the sum of discounted rewards from all future timesteps until the end of the episode. In Chapter 5 of Sutton and Barto, returns must be used to estimate the state-value and action-value functions because episode lengths are unrestricted and may be <em>greater than one</em>. In contrast, Chapter 2 deals with the very special case of multi-armed bandits in which episode lengths are always <em>equal to one</em>: The agent begins each episode in a fixed start state, takes an action, receives a reward, and then the episode terminates and the agent begins the next episode at the same start state. Therefore, <strong>a return is equivalent to a reward in Chapter 2 because all episodes have length one</strong>.</p>

CC BY-SA 4.0
2020-06-05T17:34:33.320Z
2020-06-05T17:34:33.320Z
DeepQZero
<p><strong>Return</strong> refers to the <strong>total discounted reward</strong>, starting from the current timestep.</p>

CC BY-SA 4.0
2020-06-04T04:22:51.343Z
2020-06-04T04:22:51.343Z
stoic-santiago
<p>I am reading Sutton and Barto's book on reinforcement learning. I thought that reward and return were the same things. </p>

<p>However, in Section 5.6 of the book, 3rd line, first paragraph, it is written:</p>

<blockquote>
  <p>Whereas in Chapter 2 we averaged rewards, in Monte Carlo methods we average returns.</p>
</blockquote>

<p>What does it mean? Are rewards and returns different things?</p>

<reinforcement-learning><comparison><rewards><return>
Is there any difference between reward and return in reinforcement learning?
SJa
# Is there any difference between reward and return in reinforcement learning?

(Asked by: username_0 on 2020-06-04)

I am reading Sutton and Barto's book on reinforcement learning. I thought that reward and return were the same things.

However, in Section 5.6 of the book, 3rd line, first paragraph, it is written:

> Whereas in Chapter 2 we averaged rewards, in Monte Carlo methods we average returns.

What does it mean? Are rewards and returns different things?

**Tags:** reinforcement-learning, comparison, rewards, return


---

## 2 answers

### Answer by username_2 on 2020-06-04. Score: 6 | Accepted answer

**Return** refers to the **total discounted reward**, starting from the current timestep.


---

### Answer by username_1 on 2020-06-05. Score: 3

As the accepted answer states, the return at the current timestep is equal to the sum of discounted rewards from all future timesteps until the end of the episode. In Chapter 5 of Sutton and Barto, returns must be used to estimate the state-value and action-value functions because episode lengths are unrestricted and may be *greater than one*. In contrast, Chapter 2 deals with the very special case of multi-armed bandits in which episode lengths are always *equal to one*: The agent begins each episode in a fixed start state, takes an action, receives a reward, and then the episode terminates and the agent begins the next episode at the same start state. Therefore, **a return is equivalent to a reward in Chapter 2 because all episodes have length one**.


---

