We present a meta-learning method for learning parametric loss functions that can generalize across different tasks and model architectures. We develop a pipeline for training such loss functions, targeted at maximizing the performance of model learning with them. We observe that the loss landscape produced by our learned losses significantly improves upon the original task-specific losses in both supervised and reinforcement learning tasks. Furthermore, we show that our meta-learning framework is flexible enough to incorporate additional information at meta-train time. This information shapes the learned loss function such that the environment does not need to provide this information during meta-test time.
This paper proposes a mete-learning approach to learn a loss function from old tasks which can generalize well to new tasks. The benefits of the proposed approaches are 1) data-driven a loss function and 2) allowing the usage of extra side-information to design the loss function.
Overall, the presentation in this paper is hard for me to understand technical details and see the difference with existing methods. Please see the questions below. I am glad to discuss problems with the authors' reply during the rebuttal.
Q1. "Mφ(y; fθ(x)) that predicts the loss gave the ground truth target y and the predicted target fθ(x)", "the purpose...loss function, or a meta-loss, which generalizes across multiple training contexts or tasks".
- It is better for the authors to visualize all y v.s. f_{\theta}(x) for all experiments. They have done this only for "Section 4.2.1 SHAPING LOSS FOR REGRESSION". It is better to do this for all the experiments. In this way, we can see why meta-loss can better.
- Besides, it is also better to show some example samples where the learning loss is significantly different from the human-designed one. This helps the reader better understand why the meta-loss can better.
- Finally, the authors claim the extra-information used in the meta-training is helpful. How can we see this point? There is not a step-by-step ablation study on this point.
Q2. Except for problem setup, i.e., learning a loss function, what are novelties in using meta-learning techniques?
Q3. The authors present three usages of the proposed framework in Section 3. Could the authors describe one in detail and then briefly mention the other two usages instead of writing them with the same importance? In this way, readers can understand materials and novelties better. 
- For example, I do not understand how exactly gradients are updated on meta-level. The description in Section 3.1. is too brief. 
Q4. Why the convergence speed of the meta-learner is important? e.g., Figure 4(b-c).
Q5. We have some basic restrictions for "loss function", i.e., loss(x, y) >= 0 for any x, y; loss(x, x) = 0. How such basic requirements are ensured by the learned meta-loss?
Q6. Could the authors add more explanation in the experiments and motivation in the main text? Currently, the authors just describe what they have done in the proposed method and what they have observed in experiments, just a list of facts (see Q1).