Body: <p>I have a theoretical question about RL. I use Unity ml-agents, but maybe this is more a generalized thing.</p>
  <p>In Unity ml-agents there is this concept that actions in episodes that generate more overall rewards will be prioritized over actions in less reward-full episode. This kind of doesn't make sense to me, because some action in the more rewarding episode can be worse then ones in the less rewarding one (especially for long episodes). And in general an action should prioritized depending on the step-to-step reward in respect to future rewards and those discount factor (gamma), right?</p>
  <p>But why then giving tiny negative constant reward on every step can make the agent learn to end the episode faster? And why its recommended to have constant max step length for continuous tasks? And this concept of &quot;agent comparing episodes&quot; people are using?</p>
  <p>Practical example is training a car to drive different length tracks. Longer track means more rewards. But should it create a problem, because of the &quot;episode reward comparing&quot;?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-04-12T15:35:49.480Z
Comments.Text: Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer.
Comments.UserDisplayName: Community
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-04-13T20:11:07.017Z
Comments.Text: Instead of writing "Theoretical question about reinforcement learning episodes", just put your **specific question** in the title.
Comments.UserDisplayName: nbro
Tags: <reinforcement-learning><machine-learning>
Title: Theoretical question about reinforcement learning episodes
UserDisplayName: trshmanx
ThreadText: # Theoretical question about reinforcement learning episodes
  
  (Asked by: username_0 on 2023-04-07)
  
  I have a theoretical question about RL. I use Unity ml-agents, but maybe this is more a generalized thing.
  
  In Unity ml-agents there is this concept that actions in episodes that generate more overall rewards will be prioritized over actions in less reward-full episode. This kind of doesn't make sense to me, because some action in the more rewarding episode can be worse then ones in the less rewarding one (especially for long episodes). And in general an action should prioritized depending on the step-to-step reward in respect to future rewards and those discount factor (gamma), right?
  
  But why then giving tiny negative constant reward on every step can make the agent learn to end the episode faster? And why its recommended to have constant max step length for continuous tasks? And this concept of "agent comparing episodes" people are using?
  
  Practical example is training a car to drive different length tracks. Longer track means more rewards. But should it create a problem, because of the "episode reward comparing"?
  
  **Tags:** reinforcement-learning, machine-learning
  
  > **Comments:**
  > **username_1**: Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer.  
  > **username_2**: Instead of writing "Theoretical question about reinforcement learning episodes", just put your **specific question** in the title.  
  
  ---
  
  ## 0 answers
  
  