Answers.Body: <p>Yes, early Deep-Q papers set out to prove a single architecture can reasonably approximate the policy for a single Atari game but not all Atari games at the same time.</p>
  <p>In the simplest version of your question you seem to ask if a single model can generalize to a few similar tasks, not just one.  This is still within the range of vanilla DQN as possible especially if features of several games in the distribution have common elements (ie, paddle ball games, side scrolling shooter, etc).  Model sizes inevitably must increase and dissimilarity of tasks will make progress slow.</p>
  <p>As the tasks become very dissimilar we wade into the waters of Meta-RL.  You might check out &quot;A Survey of Meta-Reinforcement Learning&quot; by Beck et. al 2023 for a very recent discussion of the state of the field.</p>
  <p>If the task(s) are to be fully learned and then more tasks introduced later the field becomes continual, lifelong or non-stationary RL.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-10-13T20:21:10.273Z
Answers.LastActivityDate: 2023-10-13T20:21:10.273Z
Answers.UserDisplayName: foreverska
Body: <p>In the classic <a href="https://www.nature.com/articles/nature14236" rel="nofollow noreferrer">&quot;human level control&quot; paper</a>, it writes:</p>
  <blockquote>
  <p>We demonstrate that the deep Q-network agent, receiving only the pixels and the game score as inputs, was able to surpass the performance of all previous algorithms and achieve a level comparable to that of a professional human games tester across a set of 49 games, using the same algorithm, network architecture and hyperparameters.</p>
  </blockquote>
  <p>It seems that different networks(agents) were trained on different tasks. Is that right?</p>
  <p>Can single agent trained by RL handle total different tasks? e.g., can we train a network to get reasonable results on different Atari games?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-10-12T13:01:48.850Z
Comments.Text: multi task RL and Continual RL are two very broad areas of research
Comments.UserDisplayName: Alberto
Tags: <reinforcement-learning>
Title: Can single agent trained by RL handle total different tasks?
UserDisplayName: zhixin
ThreadText: # Can single agent trained by RL handle total different tasks?
  
  (Asked by: username_0 on 2023-10-12)
  
  In the classic ["human level control" paper](https://www.nature.com/articles/nature14236), it writes:
  
  > We demonstrate that the deep Q-network agent, receiving only the pixels and the game score as inputs, was able to surpass the performance of all previous algorithms and achieve a level comparable to that of a professional human games tester across a set of 49 games, using the same algorithm, network architecture and hyperparameters.
  
  It seems that different networks(agents) were trained on different tasks. Is that right?
  
  Can single agent trained by RL handle total different tasks? e.g., can we train a network to get reasonable results on different Atari games?
  
  **Tags:** reinforcement-learning
  
  > **Comments:**
  > **username_1**: multi task RL and Continual RL are two very broad areas of research  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2023-10-13. Score: 1 | Accepted answer
  
  Yes, early Deep-Q papers set out to prove a single architecture can reasonably approximate the policy for a single Atari game but not all Atari games at the same time.
  
  In the simplest version of your question you seem to ask if a single model can generalize to a few similar tasks, not just one. This is still within the range of vanilla DQN as possible especially if features of several games in the distribution have common elements (ie, paddle ball games, side scrolling shooter, etc). Model sizes inevitably must increase and dissimilarity of tasks will make progress slow.
  
  As the tasks become very dissimilar we wade into the waters of Meta-RL. You might check out "A Survey of Meta-Reinforcement Learning" by Beck et. al 2023 for a very recent discussion of the state of the field.
  
  If the task(s) are to be fully learned and then more tasks introduced later the field becomes continual, lifelong or non-stationary RL.
  
  
  ---
  
  