Body: <p>I would like the community to help me understand if the following example would be better represented as episodic or continuous task, this will help me structure the problem and chose the right RL algorithm.</p>
  <p>The agent start with an initial score <code>x</code> of let's say 100. The agent objective is to maximise it's score. There is no upper bound! Theoretically the agent can get a score up to infinity, and there is no termination based on the number of steps, therefore the agent could play forever. However, the score can't be negative and if the agent get to a score of zero, the episode should terminate and the environment reset. I am undecided what would be the best representation, because if the agent learns how to play, the episode would never terminate, and the agent would theoretically play forever. However if the score get to zero, there is no way for the agent to continue playing so the environment needs to reset. Thank you.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-28T11:54:51.497Z
Comments.Text: your problem is episodic as there is a terminal state. this is similar to cart pole in that a successful agent would never have an end of episode because it can perfectly balance the pole without it dropping. It may be worthwhile having an artificial stopping criteria for training — e.g. stopping after 5000 steps and restarting (but don’t count this as a termination). The number of steps would depend on your environment and how much it changes as the length of the episode goes on
Comments.UserDisplayName: David
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-28T15:19:50.083Z
Comments.Text: Thank you for your answer! The environment is quite chaotic and ever changing, and the agent should always try to adapt to it, therefore it is not quite like the cart pole in the sense that  in cart pole, once the agent has learned how to balance the pole will stay always upright. But i see your point, and i think you may be right
Comments.UserDisplayName: DarioB
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-12-28T18:35:06.370Z
Comments.Text: In which case, I would definitely recommend using some kind of TD algorithm (Actor Critic/Q-Learning) and letting episodes run as usual. Make sure to add checkpoints to save the model so you can kill it when it is sufficiently trained
Comments.UserDisplayName: David
Tags: <reinforcement-learning>
Title: Should I represent my reinforcement learning as an episodic or continuous task?
UserDisplayName: DarioB
ThreadText: # Should I represent my reinforcement learning as an episodic or continuous task?
  
  (Asked by: username_0 on 2021-12-28)
  
  I would like the community to help me understand if the following example would be better represented as episodic or continuous task, this will help me structure the problem and chose the right RL algorithm.
  
  The agent start with an initial score `x` of let's say 100. The agent objective is to maximise it's score. There is no upper bound! Theoretically the agent can get a score up to infinity, and there is no termination based on the number of steps, therefore the agent could play forever. However, the score can't be negative and if the agent get to a score of zero, the episode should terminate and the environment reset. I am undecided what would be the best representation, because if the agent learns how to play, the episode would never terminate, and the agent would theoretically play forever. However if the score get to zero, there is no way for the agent to continue playing so the environment needs to reset. Thank you.
  
  **Tags:** reinforcement-learning
  
  > **Comments:**
  > **username_1**: your problem is episodic as there is a terminal state. this is similar to cart pole in that a successful agent would never have an end of episode because it can perfectly balance the pole without it dropping. It may be worthwhile having an artificial stopping criteria for training — e.g. stopping after 5000 steps and restarting (but don’t count this as a termination). The number of steps would depend on your environment and how much it changes as the length of the episode goes on  
  > **username_0**: Thank you for your answer! The environment is quite chaotic and ever changing, and the agent should always try to adapt to it, therefore it is not quite like the cart pole in the sense that  in cart pole, once the agent has learned how to balance the pole will stay always upright. But i see your point, and i think you may be right  
  > **username_1**: In which case, I would definitely recommend using some kind of TD algorithm (Actor Critic/Q-Learning) and letting episodes run as usual. Make sure to add checkpoints to save the model so you can kill it when it is sufficiently trained  
  
  ---
  
  ## 0 answers
  
  