Answers.Body: <p>RL can be used for cases where you have <strong>sparse rewards</strong> (i.e. at almost every step all rewards are zero), but, in such a setting, the experience the agent receives during the trajectory does not provide much information regarding the quality of the actions.</p>
  <p>Games can be often formulated as episodic tasks. For example, you could formulate a chess match as an episode and you could give a (non-zero) reward only at the end of the match. However, in this specific case, it will be hard for the RL to understand which moves have mainly contributed to the reward received, which is known as the <a href="https://ai.stackexchange.com/q/12908/2444">credit assignment problem</a>.</p>
  <p>You can solve the issue of sparse rewards with <strong>reward shaping</strong> (in particular, <a href="https://people.eecs.berkeley.edu/%7Epabbeel/cs287-fa09/readings/NgHaradaRussell-shaping-ICML1999.pdf" rel="noreferrer">potential-reward shaping</a>).</p>
  <p>The term &quot;delayed rewards&quot; may also refer to the cases where you receive only one reward at the end of the episode, although it may more usually refer to scenarios where the reward at one-time step is only received later (for some reason).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-05-07T15:59:34.377Z
Answers.LastActivityDate: 2020-11-03T13:37:38.540Z
Answers.LastEditDate: 2020-11-03T13:37:38.540Z
Answers.UserDisplayName: nbro
Body: <p>Is reinforcement learning problem adaptable to the setting when there is only one - final - reward. I am aware of problems with sparse and delayed rewards, but what about only one reward and a quite long path?</p>
  
Tags: <reinforcement-learning><reward-design><sparse-rewards><credit-assignment-problem><delayed-rewards>
Title: Can reinforcement learning be used for tasks where only one final reward is received?
UserDisplayName: TomR
ThreadText: # Can reinforcement learning be used for tasks where only one final reward is received?
  
  (Asked by: username_0 on 2019-05-07)
  
  Is reinforcement learning problem adaptable to the setting when there is only one - final - reward. I am aware of problems with sparse and delayed rewards, but what about only one reward and a quite long path?
  
  **Tags:** reinforcement-learning, reward-design, sparse-rewards, credit-assignment-problem, delayed-rewards
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-05-07. Score: 14 | Accepted answer
  
  RL can be used for cases where you have **sparse rewards** (i.e. at almost every step all rewards are zero), but, in such a setting, the experience the agent receives during the trajectory does not provide much information regarding the quality of the actions.
  
  Games can be often formulated as episodic tasks. For example, you could formulate a chess match as an episode and you could give a (non-zero) reward only at the end of the match. However, in this specific case, it will be hard for the RL to understand which moves have mainly contributed to the reward received, which is known as the [credit assignment problem](https://ai.stackexchange.com/q/12908/2444).
  
  You can solve the issue of sparse rewards with **reward shaping** (in particular, [potential-reward shaping](https://people.eecs.berkeley.edu/%7Epabbeel/cs287-fa09/readings/NgHaradaRussell-shaping-ICML1999.pdf)).
  
  The term "delayed rewards" may also refer to the cases where you receive only one reward at the end of the episode, although it may more usually refer to scenarios where the reward at one-time step is only received later (for some reason).
  
  
  ---
  
  