Answers.Body: <p>This depends on the setting. Ongoing learning that never ends is a feature of settings where one or both of the following is true:</p>
  <ul>
  <li><p>There is little existing available data or experience when considering the complexity of the problem, and it is practical to deploy a partially-trained model in the real environment (e.g. it is better than nothing, even partially trained) and continue to train it.</p>
  </li>
  <li><p>The properties of the environment change over time, so that existing models degrade in performance unless updated with new data or experience.</p>
  </li>
  </ul>
  <p>There is no major difference between reinforcement learning (RL) and supervised learning in this regard. RL lends itself well to changing environments though, because it must already be designed to adjust for changes to a policy as it progressively improves.</p>
  <blockquote>
  <p>I don't understand if the purposes of RL agents is simply optimizing a model with a reward</p>
  </blockquote>
  <p>It is still a common scenario to have separate training and test/deployment phases when developing an RL agent.</p>
  <p>If you want to have an agent that can play a specific board game or video game, or to navigate a robot in a building, it is often possible to spend enough training time that data availability is not an issue, and to generalise about all the variations of the environment that need to be handled. In which case it can be preferable to have an agent with a known, measured ability that won't change.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-23T07:03:33.990Z
Answers.LastActivityDate: 2022-07-23T07:03:33.990Z
Answers.UserDisplayName: Neil Slater
Answers.Body: <p>I think it is better to answer this question based on the reference book <em>&quot;Reinforcement Learning: An Introduction by Andrew Barto and Richard S. Sutton&quot;</em></p>
  <p>At first, come to see a general description of RL:</p>
  <blockquote>
  <p>Reinforcement learning is learning what to do—how to map situations to
  actions—so as to maximize a numerical reward signal. The learner is
  not told which actions to take, but instead must discover which
  actions yield the most reward by trying them. In the most interesting
  and challenging cases, actions may affect not only the immediate
  reward but also the next situation and, through that, all subsequent
  rewards. These two characteristics—trial-and-error search and delayed
  reward—are the two most important distinguishing features of
  reinforcement learning.</p>
  </blockquote>
  <p>Also, there is a simple comparison with supervised learning:</p>
  <blockquote>
  <p>Reinforcement learning is different from supervised learning, the kind
  of learning studied in most current research in the field of machine
  learning. Supervised learning is learning from a training set of
  labeled examples provided by a knowledgable external supervisor. Each
  example is a description of a situation together with a
  specification—the label—of the correct action the system should take
  to that situation, which is often to identify a category to which the
  situation belongs. The object of this kind of learning is for the
  system to extrapolate, or generalize, its responses so that it acts
  correctly in situations not present in the training set. This is an
  important kind of learning, but alone it is not adequate for learning
  from interaction. In interactive problems it is often impractical to
  obtain examples of desired behavior that are both correct and
  representative of all the situations in which the agent has to act. In
  uncharted territory—where one would expect learning to be most
  beneficial—an agent must be able to learn from its own experience.</p>
  </blockquote>
  <p>We can tell that the RL approach provides a straightforward adaption for the agent. Thus, the agent only needs the training to be a master of a skill.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-07-22T20:27:07.423Z
Answers.LastActivityDate: 2022-07-22T20:27:07.423Z
Answers.UserDisplayName: Pouyan
Body: <p>I don't understand if the purposes of RL agents is simply optimizing a model with a reward instead of using labeled data (i.e. in a supervision fashion), or they have also the purpose of keep training and exploring in order to adapt to possible environment changes.</p>
  
Tags: <reinforcement-learning><deep-rl><training><exploration-exploitation-tradeoff>
Title: Are RL algorithms suppose to keep learning?
UserDisplayName: pippo
ThreadText: # Are RL algorithms suppose to keep learning?
  
  (Asked by: username_0 on 2022-07-22)
  
  I don't understand if the purposes of RL agents is simply optimizing a model with a reward instead of using labeled data (i.e. in a supervision fashion), or they have also the purpose of keep training and exploring in order to adapt to possible environment changes.
  
  **Tags:** reinforcement-learning, deep-rl, training, exploration-exploitation-tradeoff
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2022-07-23. Score: 2
  
  This depends on the setting. Ongoing learning that never ends is a feature of settings where one or both of the following is true:
  
  * There is little existing available data or experience when considering the complexity of the problem, and it is practical to deploy a partially-trained model in the real environment (e.g. it is better than nothing, even partially trained) and continue to train it.
  * The properties of the environment change over time, so that existing models degrade in performance unless updated with new data or experience.
  
  There is no major difference between reinforcement learning (RL) and supervised learning in this regard. RL lends itself well to changing environments though, because it must already be designed to adjust for changes to a policy as it progressively improves.
  
  > I don't understand if the purposes of RL agents is simply optimizing a model with a reward
  
  It is still a common scenario to have separate training and test/deployment phases when developing an RL agent.
  
  If you want to have an agent that can play a specific board game or video game, or to navigate a robot in a building, it is often possible to spend enough training time that data availability is not an issue, and to generalise about all the variations of the environment that need to be handled. In which case it can be preferable to have an agent with a known, measured ability that won't change.
  
  
  ---
  
  ### Answer by username_2 on 2022-07-22. Score: 0
  
  I think it is better to answer this question based on the reference book *"Reinforcement Learning: An Introduction by Andrew Barto and Richard S. Sutton"*
  
  At first, come to see a general description of RL:
  
  > Reinforcement learning is learning what to do—how to map situations to
  > actions—so as to maximize a numerical reward signal. The learner is
  > not told which actions to take, but instead must discover which
  > actions yield the most reward by trying them. In the most interesting
  > and challenging cases, actions may affect not only the immediate
  > reward but also the next situation and, through that, all subsequent
  > rewards. These two characteristics—trial-and-error search and delayed
  > reward—are the two most important distinguishing features of
  > reinforcement learning.
  
  Also, there is a simple comparison with supervised learning:
  
  > Reinforcement learning is different from supervised learning, the kind
  > of learning studied in most current research in the field of machine
  > learning. Supervised learning is learning from a training set of
  > labeled examples provided by a knowledgable external supervisor. Each
  > example is a description of a situation together with a
  > specification—the label—of the correct action the system should take
  > to that situation, which is often to identify a category to which the
  > situation belongs. The object of this kind of learning is for the
  > system to extrapolate, or generalize, its responses so that it acts
  > correctly in situations not present in the training set. This is an
  > important kind of learning, but alone it is not adequate for learning
  > from interaction. In interactive problems it is often impractical to
  > obtain examples of desired behavior that are both correct and
  > representative of all the situations in which the agent has to act. In
  > uncharted territory—where one would expect learning to be most
  > beneficial—an agent must be able to learn from its own experience.
  
  We can tell that the RL approach provides a straightforward adaption for the agent. Thus, the agent only needs the training to be a master of a skill.
  
  
  ---
  
  