Sharing Knowledge in Multi-Task Deep Reinforcement Learning
We study the benefit of sharing representations among tasks to enable the effective use of deep neural networks in Multi-Task Reinforcement Learning. We leverage the assumption that learning from different tasks, sharing common properties, is helpful to generalize the knowledge of them resulting in a more effective feature extraction compared to learning a single task. Intuitively, the resulting set of features offers performance benefits when used by Reinforcement Learning algorithms. We prove this by providing theoretical guarantees that highlight the conditions for which is convenient to share representations among tasks, extending the well-known finite-time bounds of Approximate Value-Iteration to the multi-task setting. In addition, we complement our analysis by proposing multi-task extensions of three Reinforcement Learning algorithms that we empirically evaluate on widely used Reinforcement Learning benchmarks showing significant improvements over the single-task counterparts in terms of sample efficiency and performance.
The submission derives bounds for approximate value and policy iteration for the multitask case in reinforcement learning. In addition, two common RL algorithms are adapted to demonstrate benefits of multitask RL given related tasks. The paper is mostly well written but sometimes introduces potentially unnecessarily complex mathematical notation including missing and dual definitions which slow down the reader. Examples of missing definitions are given by K an n on top of page 3. Generelly, the paper is quite self consist thanks to minor changes in existing algorithms to show experimental results for the theoretical insights, but instead of requiring the reader to find other papers it would be better to define all terminology in the appendix (see equation 6). The main contribution of the submission is the extension of an existing bound to the multitask case as the architectures are common across existing work. The extension is relevant given growing interest in meta and multitask RL but the changes in comparison to the single task case are minor and the experiment’s value purely lies in supporting this extension.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Sharing Knowledge in Multi-Task Deep Reinforcement Learning
We study the benefit of sharing representations among tasks to enable the effective use of deep neural networks in Multi-Task Reinforcement Learning. We leverage the assumption that learning from different tasks, sharing common properties, is helpful to generalize the knowledge of them resulting in a more effective feature extraction compared to learning a single task. Intuitively, the resulting set of features offers performance benefits when used by Reinforcement Learning algorithms. We prove this by providing theoretical guarantees that highlight the conditions for which is convenient to share representations among tasks, extending the well-known finite-time bounds of Approximate Value-Iteration to the multi-task setting. In addition, we complement our analysis by proposing multi-task extensions of three Reinforcement Learning algorithms that we empirically evaluate on widely used Reinforcement Learning benchmarks showing significant improvements over the single-task counterparts in terms of sample efficiency and performance.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



The submission derives bounds for approximate value and policy iteration for the multitask case in reinforcement learning. In addition, two common RL algorithms are adapted to demonstrate benefits of multitask RL given related tasks. The paper is mostly well written but sometimes introduces potentially unnecessarily complex mathematical notation including missing and dual definitions which slow down the reader. Examples of missing definitions are given by K an n on top of page 3. Generelly, the paper is quite self consist thanks to minor changes in existing algorithms to show experimental results for the theoretical insights, but instead of requiring the reader to find other papers it would be better to define all terminology in the appendix (see equation 6). The main contribution of the submission is the extension of an existing bound to the multitask case as the architectures are common across existing work. The extension is relevant given growing interest in meta and multitask RL but the changes in comparison to the single task case are minor and the experiment’s value purely lies in supporting this extension.
assistant
The submission derives bounds for approximate value and policy iteration for the multitask case in reinforcement learning. In addition, two common RL algorithms are adapted to demonstrate benefits of multitask RL given related tasks. The paper is mostly well written but sometimes introduces potentially unnecessarily complex mathematical notation including missing and dual definitions which slow down the reader. Examples of missing definitions are given by K an n on top of page 3. Generelly, the paper is quite self consist thanks to minor changes in existing algorithms to show experimental results for the theoretical insights, but instead of requiring the reader to find other papers it would be better to define all terminology in the appendix (see equation 6). The main contribution of the submission is the extension of an existing bound to the multitask case as the architectures are common across existing work. The extension is relevant given growing interest in meta and multitask RL but the changes in comparison to the single task case are minor and the experiment’s value purely lies in supporting this extension.