Adapting Auxiliary Losses Using Gradient Similarity
One approach to deal with the statistical inefficiency of neural networks is to rely on auxiliary losses that help to build useful representations. However, it is not always trivial to know if an auxiliary task will be helpful for the main task and when it could start hurting. We propose to use the cosine similarity between gradients of tasks as an adaptive weight to detect when an auxiliary loss is helpful to the main loss. We show that our approach is guaranteed to converge to critical points of the main task and demonstrate the practical usefulness of the proposed algorithm in a few domains: multi-task supervised learning on subsets of ImageNet, reinforcement learning on gridworld, and reinforcement learning on Atari games.
 The paper proposes a method for using auxiliary tasks to support the optimization with respect to a main task. In particular, the method assumes the existence of a loss function for the main task that we are interested in, and a loss function for an auxiliary task that shares at least some of the parameters with the main loss function. When optimizing for the main loss function, the gradient of the auxiliary loss function is also used to update the shared parameters in cases of high cosine similarity with the main task. The method is demonstrated on image classification and a few reinforcement learning settings. The idea of the paper is simple, and the method has a nice property of (if ignoring some caveats) guaranteeing steps that are directionally correct with respect to the main task. In that sense it is useful in practice, as it limits the potential damage the auxiliary task does to the optimization of the main task. As the authors also note, the method suffers from some drawbacks. Although the method limits the negative effect of the auxiliary task on the optimization of the main loss function, it can still slow down optimization if the auxiliary task is not well chosen. In that sense, the method is no silver bullet. In addition, the method seems fairly computationally expensive (it would be interesting to understand how much it slows down an update, I would assume the added complexity is roughly a constant multiplier). However, as an alternative to naively adding an auxiliary task, the proposed method is a welcome addition to the tool box of practitioners. Although the experiments presented in the paper are quite different from each other, I would have wished for even more experiments. The reason is that as the method does not guarantee faster convergence, its applicability is mainly an empirical question. Especially experiments where auxiliary tasks have been used before would be interesting to test with the only addition being introducing the method proposed. The paper is generally well written and the results are fairly clearly presented. As a minor comment, the authors might want to check that articles (such as "the") are not missing in the text. All in all, the main merit of the proposed method is its conceptual simplicity and easy to understand value in practical applications where an auxiliary loss function is available. The method also seems to work well enough in the experiments presented.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Adapting Auxiliary Losses Using Gradient Similarity
One approach to deal with the statistical inefficiency of neural networks is to rely on auxiliary losses that help to build useful representations. However, it is not always trivial to know if an auxiliary task will be helpful for the main task and when it could start hurting. We propose to use the cosine similarity between gradients of tasks as an adaptive weight to detect when an auxiliary loss is helpful to the main loss. We show that our approach is guaranteed to converge to critical points of the main task and demonstrate the practical usefulness of the proposed algorithm in a few domains: multi-task supervised learning on subsets of ImageNet, reinforcement learning on gridworld, and reinforcement learning on Atari games.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 The paper proposes a method for using auxiliary tasks to support the optimization with respect to a main task. In particular, the method assumes the existence of a loss function for the main task that we are interested in, and a loss function for an auxiliary task that shares at least some of the parameters with the main loss function. When optimizing for the main loss function, the gradient of the auxiliary loss function is also used to update the shared parameters in cases of high cosine similarity with the main task. The method is demonstrated on image classification and a few reinforcement learning settings. The idea of the paper is simple, and the method has a nice property of (if ignoring some caveats) guaranteeing steps that are directionally correct with respect to the main task. In that sense it is useful in practice, as it limits the potential damage the auxiliary task does to the optimization of the main task. As the authors also note, the method suffers from some drawbacks. Although the method limits the negative effect of the auxiliary task on the optimization of the main loss function, it can still slow down optimization if the auxiliary task is not well chosen. In that sense, the method is no silver bullet. In addition, the method seems fairly computationally expensive (it would be interesting to understand how much it slows down an update, I would assume the added complexity is roughly a constant multiplier). However, as an alternative to naively adding an auxiliary task, the proposed method is a welcome addition to the tool box of practitioners. Although the experiments presented in the paper are quite different from each other, I would have wished for even more experiments. The reason is that as the method does not guarantee faster convergence, its applicability is mainly an empirical question. Especially experiments where auxiliary tasks have been used before would be interesting to test with the only addition being introducing the method proposed. The paper is generally well written and the results are fairly clearly presented. As a minor comment, the authors might want to check that articles (such as "the") are not missing in the text. All in all, the main merit of the proposed method is its conceptual simplicity and easy to understand value in practical applications where an auxiliary loss function is available. The method also seems to work well enough in the experiments presented.
assistant
 The paper proposes a method for using auxiliary tasks to support the optimization with respect to a main task. In particular, the method assumes the existence of a loss function for the main task that we are interested in, and a loss function for an auxiliary task that shares at least some of the parameters with the main loss function. When optimizing for the main loss function, the gradient of the auxiliary loss function is also used to update the shared parameters in cases of high cosine similarity with the main task. The method is demonstrated on image classification and a few reinforcement learning settings. The idea of the paper is simple, and the method has a nice property of (if ignoring some caveats) guaranteeing steps that are directionally correct with respect to the main task. In that sense it is useful in practice, as it limits the potential damage the auxiliary task does to the optimization of the main task. As the authors also note, the method suffers from some drawbacks. Although the method limits the negative effect of the auxiliary task on the optimization of the main loss function, it can still slow down optimization if the auxiliary task is not well chosen. In that sense, the method is no silver bullet. In addition, the method seems fairly computationally expensive (it would be interesting to understand how much it slows down an update, I would assume the added complexity is roughly a constant multiplier). However, as an alternative to naively adding an auxiliary task, the proposed method is a welcome addition to the tool box of practitioners. Although the experiments presented in the paper are quite different from each other, I would have wished for even more experiments. The reason is that as the method does not guarantee faster convergence, its applicability is mainly an empirical question. Especially experiments where auxiliary tasks have been used before would be interesting to test with the only addition being introducing the method proposed. The paper is generally well written and the results are fairly clearly presented. As a minor comment, the authors might want to check that articles (such as "the") are not missing in the text. All in all, the main merit of the proposed method is its conceptual simplicity and easy to understand value in practical applications where an auxiliary loss function is available. The method also seems to work well enough in the experiments presented.