One approach to deal with the statistical inefficiency of neural networks is to rely on auxiliary losses that help to build useful representations. However, it is not always trivial to know if an auxiliary task will be helpful for the main task and when it could start hurting. We propose to use the cosine similarity between gradients of tasks as an adaptive weight to detect when an auxiliary loss is helpful to the main loss. We show that our approach is guaranteed to converge to critical points of the main task and demonstrate the practical usefulness of the proposed algorithm in a few domains: multi-task supervised learning on subsets of ImageNet, reinforcement learning on gridworld, and reinforcement learning on Atari games.
The paper is addressing the problem of a specific multi-task learning setup such that there are two tasks namely main task and auxiliary task. Auxiliary task is used for the sole purpose of helping the main one. In other words, auxiliary task performance is not of interest. The simple and sensible approach proposed in the paper is using cosine similarity between the gradients of two loss functions and incorporating the auxiliary one if it is positively aligned with the main gradient. Authors suggest to further scale loss functions using the cosine similarity but it only experiments with the simpler case of binary decision of using both gradients or only the main one. Authors provide a convergence guarantee (without any convergence rate) by simply extending the convergence of gradient method.
The paper is definitely addressing an important problem as the authors cite many previous work which uses the setup of set of auxiliary tasks helping a main one. The method is simple and easy to implement. Hence, it has a potential to be useful for the community.
One major issue for me is the experimental setup. The authors cite many interesting, realistic and practical setups (Zhang et al., 2016; Jaderberg et al., 2017; Mirowski et al., 2017; Papoudakis et al., 2018), but do not use any of these setups in their experiments. Instead, paper uses set of toy experiments. This is very puzzling to me as all these papers set existing baselines for interesting problems which authors can easily compare. I think the paper needs to be experimented and compared with these established methods.
Another major issue is the weak multi-task learning baseline used in the paper. There have been many interesting developments in adaptive scaling of multiple loss functions in the literature. However, paper does not compare with them. Example of these methods are: [GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks, ICML 2018] and [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics, CVPR 2018]. Although these methods addresses the case of all tasks being important, it is a valid baseline and need to be compared. Similar to my first points, these papers also use very realistic and interesting experiments which would fit better than the toy experiments in the paper.
Final major issue is the fact that experimental results are suggesting the method is not effective. In ImageNet experiment, auxiliary tasks actually hurt the final performance as the single task is better than all methods including the proposed one. Proposed method does not guarantee that auxiliary tasks will have no harm. The GridWorld experiment is sort of a sanity check to me as it is very hand-crafted. For Breakout experiment, single task actually outperforms all baselines and this means the proposed method results in a harm similar to ImageNet case. For Breakout+MSPacMan experiment, multi task and the proposed method performs almost exactly same. I do not get why the performance on Breakout is relevant for this case since it is not a main task. The paper clearly states that only performance of an interest is the main one which is MSPacMan in this case. Also, in this experiment clearly all methods are still learning as the curve did not plateau yet. I am curious, why the learning is stop there. I do not think we need the method to be effective to be published; but, the negative result should be explained properly.
MINOR NITPICKS
- Algorithm 1&2 are crucial to understand the paper, they should be in main text
- ImageNet class IDs change between years. So, actual wordnet IDs or class names is a better thing to state
- What happens if there are multiple auxiliary tasks?
- Does the theory still hold for loss functions which are not Lipschitz as the Cauchy's gradient method requires that for convergence
In summary, the paper is proposing a sensible method for an important problem. However, it is only tested for toy problems although there are interesting existing setups which would be ideal for the method to be tested. Moreover, it is only compared with the most-naive multi task learning baselines. Even this limited experimental setup does not confirm what the paper is claiming (using auxiliary tasks only when they help). And the paper fails to explain this failure cases. The method needs to be experimented with a more realistic setup with more realistic baselines.
------
After rebuttal:
I gave detailed responses to each part of the rebuttal below. Here is the summary:
Although the response addresses some of my concerns. There are still major issues with the experimental study. 1) there are existing, relevant and well-studied multi-task setups with negative interference. Method should be experimented with some of those setups. 2) Multi-task baseline in the paper is naive and far from state-of-the-art. Paper need strong baselines as discussed. Hence, I am keeping my score. Paper needs to be improved with a stronger experimental study and need to be re-submitted.