Neural network pruning is a popular technique for reducing inference costs by removing connections, neurons, or other structure from the network. In the literature, pruning typically follows a standard procedure: train the network, remove unwanted structure (pruning), and train the resulting network further to recover accuracy (fine-tuning).
In this paper, we explore an alternative to fine-tuning: rewinding. Rather than continuing to train the resultant pruned network (fine-tuning), rewind the remaining weights to their values from earlier in training, and re-train the resultant network for the remainder of the original training process. We find that this procedure, which repurposes the strategy for finding lottery tickets presented by , makes it possible to prune networks further than is possible with finetuning for a given target accuracy, provided that the weights are rewound to a suitable point in training. We also find that there is a wide range of suitable rewind points that achieve higher accuracy than fine-tuning across all tested networks. Based on these results, we argue that practitioners should explore rewinding as an alternative to fine-tuning for neural network pruning.

*Summary*
Extending the observations of Frankle et al. (2019, "The Lottery Ticket Hypothesis"), this paper examines "rewinding" as an alternative to fine-tuning in a typical network pruning process. After training to convergence for T iterations, the k% of weights with the smallest magnitude are pruned (set to zero). Typically, in fine-tuning, the remaining weights are continued to be trained for several more iterations at a small learning rate. With "rewinding", the remaining weights are reset to their values in iteration 0<=t<T and are trained for T-t iterations with the original learning rate schedule.
Experiments with this method (and an iterative variant) show that (a) rewinding achieves 2-5x smaller networks matching the unpruned accuracy (with comprehensive tuning of the rewinding parameter t); (b) selection of rewinding iteration is important, but is flexible within a large range for higher sparsity pruning.
*Rating*
The paper is very well written with good exposition, thorough notes and citations for all methodological choices,
and an explicit statement of the limitations of the work.
A few considerations relevant to the rating:
(1) Novelty: The idea for rewinding is not novel, as acknowledged clearly in the paper. Frankle et al. (2019, "Stabilizing the Lottery Ticket Hypothesis") showed that rewinding to an early iteration of training (0<t<T) yielded better accuracy than rewinding to iteration t=0 for VGG-19 and ResNet-18 on CIFAR-10. However, Frankle et al. did not consider fine-tuning as it was not relevant to lottery ticket discovery. This submission studies the tradeoffs of rewinding vs. fine-tuning.
(2) Thoroughness: The paper considers ResNet-20 and VGG-16 with CIFAR-10 and ResNet-50 with ImageNet. Conclusions would be strengthened with additional combinations of networks and datasets.
(3) Acknowledged limitations: As noted, the paper doesn't consider any pruning criteria other than weight magnitude, nor does it consider structured pruning. The latter in particular is important for applications where prediction speed on commodity hardware is a limiting factor.
As it is, I think this paper a worthy (if limited) contribution to the understanding of network pruning.
*Notes*
Table 1/Figures *: note which dataset is used for each architecture
Figures 2-3: It seems that many values are clipped by the legend range of +/- 0.5%. Consider showing the figure with a larger range or adding such a figure to the appendix.


Recently, there has been a growing interest in automatically exploring neural network architecture design space with the goal of finding an architecture that improves performance (characterized as improved accuracy, speed of training, or resource requirements). However, our theoretical understanding of how model architecture affects performance or accuracy is limited. In this paper, we study the impact of model architecture on the speed of training in the context of gradient descent optimization. We use the ideas from prior work that shows gradient descent can be modeled as a first-order ODE and use ODE's coefficient matrix H to characterize the convergence rate. We introduce a simple analysis technique that enumerates H in terms of all possible "paths" in the network. We show that changes in model architecture parameters reflect as changes in the number of paths and the properties of each path, which jointly control the speed of convergence. We believe our analysis technique is useful in reasoning about more complex model architecture modifications.

This paper considers the problem of understanding the impact of deep neural networks (DNN) model architecture on the convergence rate of gradient descent dynamics. To achieve this goal, the paper follows the recent trend of continuous-time perspective of optimization, and proposes to model gradient descent via the gradient flow, which is a first-order ODE. The induced loss dynamics is then also following a first-order ODE with a coefficient matrix H (that depends on the solution trajectory, and hence non-constant). The paper then claims that the convergence rate is characterized by the minimum eigenvalue of H, and analyzes this H through a straightforward path-based formula obtained by chain rules. In particular, the authors try to explain the effect of width, depth and number of paths on the convergence rate, and validated these through a few numerical experiments.
Admittedly, the idea of this paper is interesting. However, I think the novelty and rigorousness of this paper is not convincing, as explained in more details below.
1. On the novelty side, characterizing the convergence via the H matrix is not new, and most of the discussions in Section 4 have appeared in exactly the same form in the previous works [13] (two-layer) and [25] (general), which are also cited in this paper. In addition, the path formula is also very closely related (if not completely the same as) with the expansion of G matrix in Section 4 of [25] (where G is the H in this paper), which decomposes the G (or H) matrix into summation over layers. These facts largely lower the contribution of this paper on a high level.
2. On the rigorousness side, the paper is not very consistent in the notation.
1) The notation is not very consistent. The authors use H in the notation section to denote the matrix, use X_i to denote the data input, but use \bf{H} and x_i (lower cased) subsequently.
2) In Section 4, the authors immediately start with the continuous-time perspective, without even mentioning the gradient dynamics or some related ansatz. The authors may want to mention that they use the ansatz w_k=W(kh), where W is a smooth curve, and take h to 0 to obtain the ODE models, as is done in [1].
3) The notation list at the beginning of Section 4.3 is too long and not clear. In particular, l(p) is not even defined before appearing in (14), and \sigma_s seems to be overridden by the notation activation(w_s, X_i) and does not appear later in the path gradient, which is weird. Shouldn't there be \sigma_s inside the formula of (14)?
3. Again on the rigorousness side, the paper is very non-rigorous when stating the claims and theorems.
1) The authors claim that "This result will hold for other l_p losses", but indeed what holds is different for l_p losses (the rate is scaled by p/2).
2) Section 4.2 does not make any sense. The authors should either directly cite the corresponding content in [25], which are much clearer, or directly invoke the standard linear ODE theory and use the matrix exponential and Taylor expansion to make the explanations.
3) The paper seems to use a very informal argument that H stays close to its initial value to derive all the theory, which is only empirically checked in the appendix. But given the proportion of the theory part of this paper, I think the paper should either clearly state the assumption as H being constant, or follow the manner of [25] to prove that H stays close to some fixed matrix and use this to prove the other theorems rigorously. Otherwise, the theory part is both hard to understand and verify.
4) Proposition 5.1 should clearly state whether the statement is in the expectation sense, or high probability sense, or something else.
5) The explanation in Section 5.2 is rather unclear. In particular, I don't understand why "eigenvalues of H(g) are pushed in the direction of g" implies that "the updates prefer the direction of u", and why this then further implies something related to the momentum acceleration. The authors should provide a rigorous statement here.
6) The authors claim at the bottom of page 1 that they show adding a new layer leads to H(t) being decomposed into an adaptive learning rate term and a momentum term. But the adaptive learning rate part is not showing anywhere later in the paper.
Minor comments:
1) In equation (5), there should be a minus sign on the right-hand side.
2) In Section 5.2, there is no "Equation 5.2". It should be something else.
3) The authors may want to add citations to [2] (which is a concurrent work with [25] on essentially the same topic) and [3] (which is a predecessor work of neural ODE).
[1] Su, Weijie, Stephen Boyd, and Emmanuel Candes. "A differential equation for modeling Nesterov’s accelerated gradient method: Theory and insights." Advances in Neural Information Processing Systems. 2014.
[2] Allen-Zhu, Zeyuan, Yuanzhi Li, and Zhao Song. "A convergence theory for deep learning via over-parameterization." arXiv preprint arXiv:1811.03962 (2018).
[3] Lu, Yiping, et al. "Beyond finite layer neural networks: Bridging deep architectures and numerical differential equations." arXiv preprint arXiv:1710.10121 (2017).
################## post rebuttal ##################
After reading the authors' rebuttal, I decide to raise my rating to 3 (weak reject).


In seeking for sparse and efficient neural network models, many previous works investigated on enforcing 1 or 0 regularizers to encourage weight sparsity during training. The 0 regularizer measures the parameter sparsity directly and is invariant to the scaling of parameter values. But it cannot provide useful gradients and therefore requires complex optimization techniques. The 1 regularizer is almost everywhere differentiable and can be easily optimized with gradient descent. Yet it is not scale-invariant and causes the same shrinking rate to all parameters, which is inefficient in increasing sparsity. Inspired by the Hoyer measure (the ratio between 1 and 2 norms) used in traditional compressed sensing problems, we present DeepHoyer, a set of sparsity-inducing regularizers that are both differentiable almost everywhere and scale-invariant. Our experiments show that enforcing DeepHoyer regularizers can produce even sparser neural network models than previous works, under the same accuracy level. We also show that DeepHoyer can be applied to both element-wise and structural pruning.

The paper focuses on sparse neural networks. Typically, l1 regularization is the go-to strategy, however, it is not scale invariant. That is, all weights are affected by the regularization, not only those that are being driven to 0. l0 regularization is theoretically optimal, however, it is not smooth and has no gradients almost everywhere, so it cannot be used for training. As a compromise the paper proposes Hoyer regularization, that is the l1/l2 ratio. The Hoyer regularization has the same minima structure and leads to sparse solutions while being scale invariant, that is it does not affect all weights in the process. Additionally, the paper proposes structured Hoyer regularization. Last, it employs the said regularizations in deep networks: LeNet, AlexNet and ResNet on several datasets: MNIST, CIFAR, ImageNet.
Strengths:
+ The described method is simple, intuitive and straightforward. By applying the said regularization (~ Σ |w_i|/sqrt(Σ w_i^2)), one arrives at seemingly sparser solutions, which is verified in practice.
+ The experiments are extensive and convincing. I particularly like that the authors have used their method with complex and deep models like ResNets, on large scale datasets like ImageNet.
+ The presentation is generally clear and one can understand the paper straightaway.
Weaknesses:
+ The contributions of the paper are rather on the thin side. At the end of the day, Hoyer regularization is taken from another field (compressed sensing) and applied on deep networks. This is also witnessed by some moderate repetition in the writing, e.g., between the introduction and the related work.
+ There are some points where the paper becomes unclear. For instance, in Figure 3 what are the "other methods"?
+ In Figure 1 it is explained that the Hoyer regularization leads to minima along the axis. The gradients then push the models "rotationally". Could this lead to bad multiple local optimal problems? Is there any guarantee that any particular axis will generate better solutions than the other?
All in all, I would recommend for now weak accept. I find the work interesting and solid, although not that exciting.


In most machine learning tasks unambiguous ground truth labels can easily be acquired. However, this luxury is often not afforded to many high-stakes, realworld scenarios such as medical image interpretation, where even expert human annotators typically exhibit very high levels of disagreement with one another. While prior works have focused on overcoming noisy labels during training, the question of how to evaluate models when annotators disagree about ground truth has remained largely unexplored. To address this, we propose the discrepancy ratio: a novel, task-independent and principled framework for validating machine learning models in the presence of high label noise. Conceptually, our approach evaluates a model by comparing its predictions to those of human annotators, taking into account the degree to which annotators disagree with one another. While our approach is entirely general, we show that in the special case of binary classification, our proposed metric can be evaluated in terms of simple, closed-form expressions that depend only on aggregate statistics of the labels and not on any individual label. Finally, we demonstrate how this framework can be used effectively to validate machine learning models that we trained on two real-world tasks from medical imaging. The discrepancy ratio metric reveals what conventional metrics do not: that our models not only vastly exceed the average human performance, but even exceed the performance of the best human experts in our datasets.

Summary: This paper proposes a novel metric "discrepancy ratio" for evaluating the performance of a model where the ground truth for each data point comes from many expert-yet-imperfect annotators. The authors suggested that this problem has remained largely unexplored. The proposed metric is intuitive and is easy-to-use. The authors suggested that this metric can be used for many applications. For example, to evaluate if a model is better than the annotators on average, to evaluate the annotator, to compare between models.
========================================================
Comments on clarity:
The writing is good overall and I am able to understand the contribution and the key idea of this paper.
Questions and comments on clarity:
1. While MPD is defined with an argument (Y, Y'), the definitions annotator discrepancy, model discrepancy and discrepancy ratio ignore the inputs of the function. It might be better to keep the arguments of the function for clarity. When people want to adopt the discrepancy ratio in their work and there are many ratios to discuss, then the proposed definition can be used conveniently.
2. In Eq. (5) \sigma seems to be undefined. I believe it is a variance but it is kinder to readers to explicitly state it (I found the authors explicitly stated it in Appendix B).
3. In Eq. (8), is it intentional to have $i$ exists over m and y as written in the paper?
4. Does the discussion of the simplicity of the discrepancy ratio hold for other \delta, i.e., not the squared loss?
5. In Section 4.1 (experiment with MNIST), did the authors binarize the data or simply discuss the multi-class problem? Because this follows section 3.3 which dedicates the whole page to discuss binary classification. It may be a bit sudden to move to multi-class classification and an additional sentence to clarify this might help.
6. I could not understand what is "single annotator strictness in Figures 1d and 2d. It is not explained if I did not miss it.
7. I think we should not resize the figure/table caption. The text size of captions are much smaller than the main text.
8. Is PLAX binary classification? (I understood as yes from Figure (3a) but I think it's also nice to clarify it to avoid confusion (also for MNIST)).
9. What is the y-axis for figure 3a?
10. I think reporting the performance of the mean-per-class discrepancy ratio without defining it in the main text (it is defined in a natural language (English) in Appendix E) makes the main body of the paper not self-contained. I strongly suggest to either (1) remove mean-per-class discrepancy ratio entirely from the main body or (2) define it mathematically in the main body. Also, I don't see reporting mean-per-class discrepancy ratio adds values to a paper for current experiments because the reported mean-per-class ratio is very similar to that of discrepancy ratio (Figures 3b (almost overlap) and Table 2 (same trend and the values are very close to the discrepancy ratio for all cases)).
11. Can we use F1 to evaluate whether a model is better than the average human performance?
Minor points on clarity, which may just be a personal preference of a reviewer:
1. The authors may consider using \begin{definition} when defining three key values: the mean pairwise deviation (MPD), annotator discrepancy, and model discrepancy. So the readers can easily look up when getting lost in definitions. And it may help to make a definition more precise (e.g., given label Y, data points, MPD is defined as follows).
2. Eqs (1), (2), (3) may be easier to understand if we put a denominator out of the summation as much as we can. For example, for Eq. 2, N can be outside of the two sums, A^i(A^i-1) can be between the two sums.
========================================================
Significance:
The problem that this paper addresses is highly important. I like the idea and I am convinced that discrepancy ratio is potentially useful for the community. And it also opens many possibilities to analyze the discrepancy ratio and to find potential applications for it.
========================================================
Other comments:
On the limitation:
I am satisfied that the authors clearly discuss the limitations of the proposed method and I agree with the discussion.
On the experiments:
1. Why we use model 1.0, 0.95, 0.9 for Figures 1c, 1d, 2c, 2d but 1.0, 0.9, 0.8 for Figures 1a, 1b, 2a, 2b? Perhaps full results with many models should also be reported for completeness.
2. Isn't it more natural to show a graph that has x-axis as label swap probability in the same figure for comparison? For example, Figure 1 only report performance with respect to each measure (Naive Evaluation, Majority Vote, Relative F1, Discrepancy ratio) with respect to label swap probability. We can fix the number of annotators to 9 for relative F1 and discrepancy ratio, to validate whether it also becomes increasingly difficult to differentiate model improvements or not for them. Then after suggesting that Naive evaluation is not good and other metrics look fine. Majority vote cannot evaluate that the model is better than human average performance, which we want to know. Then Figure 2 highlights the contribution of the discrepancy ratio and shows that it outperforms relative F1 in some perspectives.
3. For the MNIST benchmark dataset, it is possible to show the performance on the test ground truth too. And I think it is interesting to compare different models with the same label noise rate and confirm that discrepancy ratio is successful to evaluate the best model with respect to the ground truth accuracy. It would be very impressive to see if the best model with respect to the discrepancy ratio can successfully be the best also on the ground truth test set.
On space usage:
I feel there are many important things to be included in the main body but it is instead in the appendix. On the other hand, I think there are several parts that can be shortened in the main body. And it seems that the binary classification example (3.3) also consumed a lot of space. The derivation may be omitted and make it a Theorem and then let's suggest the readers to read the appendix for the proof if they are interested. Then we can have more space to add important components in the main body of the paper.
========================================================
Decision:
Although there are issues on the clarity of the paper, I believe these issues are not too difficult to fix in the final version. The idea is interesting and potentially impactful. For these reasons, I vote a weak accept for this paper.
Update: I would like to thank the authors for the clarification. I have read the rebuttal.
I read through the updated version of the paper. I believe that although there is still room for improvement, the contribution of this paper is sufficient as an important step towards solving this difficult yet highly relevant problem. This paper also stimulates several future directions. The authors clearly stated the limitations of the paper, which is highly useful for researchers to improve this work or find a better solution.  The clarity of the paper is also improved. For these reasons, I increased my score from weak accept to accept.


This paper presents a sparse representation of model uncertainty for deep neural networks (DNNs) that relies on an inverse formulation of Multivariate Normal Distribution (MND): an information form. We show that the model uncertainty can be estimated in this form using a scalable Laplace Approximation scheme, which involves a diagonal correction of the Kronecker-factored eigenbasis. As this makes the inversion of the information matrix intractable -an operation that is required for a full Bayesian analysis, we further devise a novel low-rank approximation of this eigenbasis that exploits spectral sparsity of DNNs. Methods to realize this sparsification are provided that develops into a memory-wise tractable sampling computations. Both of our theoretical analysis and empirical evaluations over various benchmarks show the superiority of our approach over existing methods.

Laplace approximation has been an important tool for obtaining uncertainty estimation for deterministic models. To efficiently approximate the Hessian matrix for neural networks, Ritter et. al (2018) proposes to use K-FAC . Motivated by the relatively inaccurate approximations of K-FAC, this paper proposes to improve KFAC approximations by combining eigen-basis corrections (EK-FAC, George et al, 2018) and diagonal corrections. The paper shows that the proposed method has smaller Frobenius approximate errors compared to K-FAC and EK-FAC. To further reduce the computational costs, the paper proposes a low-rank approximation by keeping only the L largest eigenvalues. Empirically, the paper demonstrates improved calibration and out-of-distribution entropies compared to previous approaches.
# Diagonal Corrections.
With the diagonal correction approximating the Fisher better, the paper shows the computation can still be conducted in the scale of W, which is similar to K-FAC. However, the method requires the diagonal correction matrix D is always positive, which might not be true. Moreover, because the computation requires D^{-1}, clipping D to a small constant will bring up stability issues. I wonder how this problem is tackled in this paper.
# Writing
The paper's notations are messy, which requires a lot of intellectual guesses to understand the conveyed idea.
1) Notations are not introduced, such as $\delta \theta$, $W_{map}^{IV}$, the MN distribution in the appendix.
2) Notations are typoed. Eq(3) $N(0, A^{-1} \otimes G^{-1}) = MN(0, G^{-1}, A^{-1})$; The bottom paragraph in Page 4; eq(8).
3) Notations are abused. In particular for $V$ and $\Lambda$ when introducing EK-FAC. The paper uses $V$ for both true eigenbasis and the EK-FAC eigenbasis. In addition, the EK-FAC part should be moved to the background section.
4) The paragraph below Corollary 1 says the author can prove the proposed method also has closer approximations in terms of the Fisher inverse. But no proofs are given.
5) Caption of Figure4.
# Laplace Approximation
For eq(4, 8, 12), The Hessian in Laplace approximations should be divided by $N$. Although the paper also mentions the scaling below those equations. But technically eq(4, 8, 12) are wrong and I don't know whether the experiments really did the scaling or not.
# Low-rank Approximation
1) It is not clear why the low-rank approximation is necessary. The computational costs are inevitable to compute the eigen-system of A and G. Why do we need the low-rank approximation after that ? K-FAC is not a computationally expensive method either.
2) I cannot understand the proofs of Lemma 2. In fact, I don't know what $ I_{1:L}^{top}, I_{1:K}^{top}$ means. More explicit formulas should be given for clarity.
3) Lemma 4 states $I_ii = (\hat{I}_{def})_ii$. Although the diagonal correction makes $I_ii = (I_{def})_ii$, the low-rank approximation makes them unequal again. Or I guess you use a different D in eq(13) from the D in eq(10) ?
# Experiments
The paper needs more experiments to validate the proposed method. Firstly, for the MNIST experiments, it is better to use the same architecture as in Ritter et al (2018) for direct comparisons. Beyond that, the adversarial attack experiment and the mis-classification uncertainty experiment in Ritter et al (2018) seem to be good choices as well.
# Overall
The paper proposes a diagonal corrected EK-FAC, achieving better approximation of the Fisher matrix. Interestingly, the paper shows that this corrections doesn't add too much computations.  However, the proposed low-rank approximation doesn't seem necessary. And the paper's notations and presentations are too messy to be an accepted paper.


This paper introduces the deep coordination graph (DCG) for collaborative multiagent reinforcement learning. DCG strikes a flexible trade-off between representational capacity and generalization by factorizing the joint value function of all agents according to a coordination graph into payoffs between pairs of agents. The value can be maximized by local message passing along the graph, which allows training of the value function end-to-end with Q-learning. Payoff functions are approximated with deep neural networks and parameter sharing improves generalization over the state-action space. We show that DCG can solve challenging predator-prey tasks that are vulnerable to the relative overgeneralization pathology and in which all other known value factorization approaches fail.

This paper introduces the deep coordination graph for collaborative multi-agent reinforcement learning aimed to solve predator-prey tasks by preventing relative overgeneralization during the exploration of agents.
In general, this paper gives a detailed and comprehensible depiction of the Introduction, Related work and Background section. However, I have two concerns about their method and experiments.
The presentation of the “Method” section is not clear enough to evaluate the contribution of this paper. Specifically:
(1). In the "Method" section, your method incorporates three ideas: 1. restricting the payoffs; 2. sharing parameters; 3. allowing generalization;
In my understanding, both idea 1 and idea 2 come from VDN [4]. Idea 3 is not implemented in this work. Then, what is your contribution?
(2). According to the announcement, the key benefit for your work is "prevent relative overgeneralization during exploration of agents". It is hard to access if the proposed method can prevent overgeneralization. Do you have some theoretical or empirical justifications?
(3). I think it would be helpful if you could give a description according to your algorithm in the appendix. It would be better to draw a diagram to show what is your model.
I think the experiments are too simple and not convincing.
(1). I notice recent multi-agent reinforcement papers ([1], [2] closely related to your work) evaluate their work on the challenging set of StarCraft II micromanagement tasks and achieved the evident result.

(2). In fig.2, why did you only compare your model with VDN [4]?
(3). In fig.3, why does the return value of QTRAN first decrease and then increase? From [3], it seems their return curve continuously grows.
(4). In fig.4, why QTRAN fail to this task?
[1]. Rashid, Tabish, et al. "QMIX: monotonic value function factorisation for deep multi-agent reinforcement learning." arXiv preprint arXiv:1803.11485 (2018).
[2]. Wang, Tonghan, et al. "Learning Nearly Decomposable Value Functions Via Communication Minimization." arXiv preprint arXiv:1910.05366 (2019).
[3]. Son, Kyunghwan, et al. "QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning." arXiv preprint arXiv:1905.05408 (2019).
[4]. Sunehag, Peter, et al. "Value-decomposition networks for cooperative multi-agent learning." arXiv preprint arXiv:1706.05296 (2017).


Differentiable Architecture Search (DARTS) has attracted a lot of attention due to its simplicity and small search costs achieved by a continuous relaxation and an approximation of the resulting bi-level optimization problem. However, DARTS does not work robustly for new problems: we identify a wide range of search spaces for which DARTS yields degenerate architectures with very poor test performance. We study this failure mode and show that, while DARTS successfully minimizes validation loss, the found solutions generalize poorly when they coincide with high validation loss curvature in the architecture space. We show that by adding one of various types of regularization we can robustify DARTS to find solutions with less curvature and better generalization properties. Based on these observations, we propose several simple variations of DARTS that perform substantially more robustly in practice. Our observations are robust across five search spaces on three image classification tasks and also hold for the very different domains of disparity estimation (a dense regression task) and language modelling.

----- Updated after rebuttal period ---
The author's detailed response effectively addressed my concerns. I am moving my score to Accept. This paper proposes an interesting systematic study of differentiable approach in NAS.
------ Original Review ----
Summary
This paper presents a systematic evaluation on top of differentiable architecture search (DARTS) algorithm and shows it usually searched an architecture with all skip-connection. It empirically reveals that the largest eigenvalue of the Hessian matrix (\lambda) of loss w.r.t. architecture parameters has a strong correlation with the generalization ability (via loss of test dataset), and shows this \lambda will first decrease but then drastically increase after a certain epoch number on 4 different search spaces. It then proposes an early-stop scheme (DARTS-ES) to stop the search before this phenomenon occurs. In addition, it proposes to use data-augmentation, path-dropping and tuning L2 regularization during the search, namely Robust-DARTS(R-DARTS), and yield constantly better results over original DARTS on 3 datasets.
Overall, the observation that the largest eigenvalue of the Hessian matrix is novel and intriguing, and the experiments are extensive and meaningful. The idea to use more search spaces for comparison is fair and performance increase demonstrates the proposed R-DARTS and DARTS-ES  are effective. Although I still have some questions regarding the detail settings, I think this paper provides a novel angle to understand the search phase of DARTS, and proposed simple but effective regularization can be beneficial to the research community using DARTS.
Main concerns:
- Problem of DARTS as a motivation
The claims of local smoothness/sharpness and generalization are related to network generalization is quite intriguing, however, only using largest eigenvalue of Hessian matrix as an indicator of this local shape does not seem to be enough. A recent paper on loss-landscape visualization [1] provides means to examine this hypothesis directly, and could the author try to provide additional visualization to support their claim? Otherwise, the paper's claim does not generalize to the local shape of the loss function, and should stays with the largest eigenvalue. It is totally okay in my perspective, but just indicates some revision to the main text and analysis of Section 4.2.
- Questions about Figure 3 experiments
How is test error computed? Is it on a batch of test-split, or the entire one? Also, which architecture is used to compute this test error? Paper mentioned, in Section 4.1, the word "final architecture", but does this refer to the super-net (the one-shot model in paper's definition), or the stand-alone model obtained via binarized architecture alphas? If latter, is this generalization error obtained from training from scratch? Or simply using the super-net parameters during the search? Since the conclusion of this plot serves as the foundation of designing R-DARTS and DARTS-ES, if the experiments are only conducted over a small set of images or the binarized model with super-net parameters, it undercuts the credibility of the conclusion, largest architectural eigenvalue, and generalization ability.
- More independent runs of experiments.
In Figure 3, validation of original DARTS, and Table 1, DARTS vs DARTS-ES, paper runs the experiment for 3 times and take the average, but for the proposed R-DARTS, it is not. Is there a reason why not scaling the experiments? I suggest the author provide results over 5 runs, like the one in Table 4 for PTB and show if the R-DARTS truly surpasses DARTS constantly. This question also applies to Figure 1, when paper claims the original DARTS found poor cell type, could this be repetitive over multiple runs?
I guess for the experiments in Table 3, it is already done since paper mentioned the reported results are the best model of searching 4 times.
- R-DARTS failed to out-perform DARTS in the original space on CIFAR-10
This is confusing, will this suggest, if tuning well, DARTS will surpass R-DARTS(L2) in other cases as well? Since this is the only setting that DARTS is built upon.
Minor comments and questions
- Using test data during search
After showing the strong correlation between the largest eigenvalue of Hessian and the network generalization error, the early-stop is natural, however, does this mean the model selection is using the test data? Or the actual test-data is never seen during the search phase of Section 4.3.
- L2 stabilizes max eigenvalue
Paper uses L2 coefficient up to 0.0243, showing constant improvement of test error while validation error drops in CIFAR-10 of Figure 11. Could the author try larger coefficients to determine when this trend will stop?
- Question about section 4.2
Performance drop due to the binarized operation (pruning step) in DARTS analysis is very interesting, I am curious how many architectures does the paper evaluate in Figure 5, when the dominant eigenvalue is smaller than 0.25? Since the conclusion is "low curvature never led to large performance drops" if the number of points is too few, it is not that convincing, especially from the plot, we see at eigenvalue = 0.5, there exists 2 architecture with >20% drop. In addition, what does each point in Figure 5 refer to? The best model (and the binarized one according to the argmax of \alpha) of one independent DARTS run or some binarized models sampled from a distribution on top of the same DARTS run (meaning only one super-net)? Is this experiment follows the setting in Figure 4?
- Figure 6, C10 S2, DARTS-ES is worse than DARTS when Drop probability = 0.6, whereas all other cases, DARTS-ES outperforms DARTS, why does this happen? Could the author comment on it?
- ScheduledDropPath in section 5.1
Does Drop-path belongs to data-augmentation techniques? It is more like a regularization in my perspective and should be grouped with 5.2.
- S1 S2... in Table 3
Does this refer to the search space? Or different random seed (mentioned )
- one-shot v.s. weight sharing model
One-shot in NAS domain is firstly introduced by Bender et al., while Pham et al. use parameter sharing. The reason to use one-shot is that all the sub-paths will have a fair chance to be trained.
- Typos
1. In section 2.1, line 4 "better.Similarly" should have space.
--- Reference ---
[1] Li et al., Visualizing the Loss Landscape of Neural Nets, arxiv'17.


In this work, we aim to solve data-driven optimization problems, where the goal is to find an input that maximizes an unknown score function given access to a dataset of input, score pairs. Inputs may lie on extremely thin manifolds in highdimensional spaces, making the optimization prone to falling-off the manifold. Further, evaluating the unknown function may be expensive, so the algorithm should be able to exploit static, offline data. We propose model inversion networks (MINs) as an approach to solve such problems. Unlike prior work, MINs scale to extremely high-dimensional input spaces and can efficiently leverage offline logged datasets for optimization in both contextual and non-contextual settings. We show that MINs can also be extended to the active setting, commonly studied in prior work, via a simple, novel and effective scheme for active data collection. Our experiments show that MINs act as powerful optimizers on a range of contextual/non-contextual, static/active problems including optimization over images and protein designs and learning from logged bandit feedback.

This paper tackles the problem of solving a black-box optimization problem where only some samples have been observed. This task requires a good model that can be both expressive and generalizable. Instead of learning only a single forward model of x -> y, this paper proposes to additionally use a mapping from y -> x. Optimizing in the space of z instead of x can be much simpler, and this should also act as a strong regularizer during training. Specifically, the paper uses a GAN that transforms [y,z] -> x, where z is stochastically sampled. This paper further proposes a reweighting scheme that interpolates between a uniform weighting and weighting the best sample so far, as well as a sampling procedure that iteratively samples points and refits a second model, which was inspired by Thompson sampling.
Pros:
 - The proposed idea of using an inverse mapping is straightward but shown to be effective. The methods to make this work, namely reweighting and the randomized labeling procedure, seem to have some amount of theory behind them, though their presentation was confusing without multiple read-throughs.
 - There are plenty of experiments across a wide array of domains, including images, 2D and 6D functions, and proteins which have a discrete representation.
 - There are some comparisons to Spearmint and a scalable BO method (DNGO), though only on the 2D and 6D functions.
Cons:
 - The proposed pieces were often difficult to follow, and there doesn't seem to be sufficient information regarding the reweighting and randomized labeling for understanding and reproducing this work (see Questions).
 - Only a few ablation experiments were carried out, and the effect of reweighting seems to only appear as a visual comparison in Figure 1. As the method seems quite different from related approaches, having more systematic comparisons would give us a better understanding of the core sources of improvement and better instigate future works.
Questions:
- After Theorem 3.1, what is g? A linear function can only change a pdf by its bias (as any multiplication only affects the normalization constant), so not sure how to interpret g. I wasn't able to find this in the Appendix either.
- When creating the augmented dataset for defining & approximating some kind of posterior, can you be more concrete about how the y (and x) values are chosen? More specifically, is this procedure approximating some kind of meaningful posterior? If the y values are sampled randomly (from some fixed prior or perhaps from the p(y) described in the reweighting section?), what makes this procedure meaningful (does it rely on the learned GAN somehow)? Seeing a simple example in 1D would be a good visualization.
- What are the different rows of Figure 1 (different initializations)? Why are the different rows so similar for MIN but not for F? Also, what were the original examples (are they closer to the F results, or the MIN results)?
- For the protein task, how did you structure the output of f^-1(x) and how did you backpropagate through the discrete variables?
- In the experiments, it wasn't explicitly clear what the "MIN without inference" setting referred to; I assume this is where a single sample from the inverse mapping GAN was used?
Additional Comments:
- When discussing the reweighted objective, the notation is a little confusing. The variables (j,k) is a different parameterization of the index i, but initially I thought k was indexing features of x. Perhaps a brief explanation of the rearrangement would make this clearer at first glance, or even remove this from the main text? (Without going through the proof, it isn't clear why this rearrangement is discussed.)
- After (1), it wasn't yet clear why this is called "model-based" optimization, as no model has been introduced yet.
- Are BO methods truly not applicable to the static setting? The static setting seems to be just a single prediction, which seems like a single step (ie. special case) of the non-static case?
- typo: "method to solve perform optimization"
- typo: Figure 1 "Obsere"


Large-scale ground truth data sets are of crucial importance for deep learning based segmentation models, but annotating per-pixel masks is prohibitively time consuming. In this paper, we investigate interactive graph-based segmentation algorithms that enforce connectivity. To be more precise, we introduce an instanceaware heuristic of a discrete Potts model, and a class-aware Integer Linear Programming (ILP) formulation that ensures global optimum. Both algorithms can take RGB, or utilize the feature maps from any DCNN, whether trained on the target dataset or not, as input. We present competitive semantic (and panoptic) segmentation results on the PASCAL VOC 2012 and Cityscapes dataset given initial scribbles. We also demonstrate that our interactive approach can reach 90.6% mIoU on VOC validation set with an overhead of just 3 correction scribbles. They are thus suitable for interactive annotation on new or existing datasets, or can be used inside any weakly supervised learning framework on new datasets.

Summary:
- key problem: efficiently leveraging scribbles as interactive supervision (at test time) for panoptic segmentation;
- contributions: 1) two algorithms leveraging scribbles via a superpixel connectivity constraint (one class-agnostic local diffusion heuristic, one class-aware with a MRF formulation), 2) experiments on PASCAL VOC 2012 and Cityscapes showing that both methods i) can achieve good performance without training data (using RGB values or pretrained features), ii) can improve the performance of a fully supervised baseline when using its probability maps as representation, and iii) can significantly improve performance beyond the state of the art on PASCAL when used interactively (90% mIoU with 3 rounds of corrective scribbles).
Recommendation: weak reject
Key reason 1: unclear novelty and relevance to ICLR.
- The paper proposes to apply two existing algorithms (Nguyen & Brown 2015, Rempfler et 2016) to a new task (interactive panoptic segmentation): what is the claimed novelty? What is specific to panoptic segmentation vs semantic or instance segmentation? Could the difference with related work in Section 2 be discussed more precisely?
- Furthermore, there seems to be no learning (representation or otherwise) involved in this submission. The paper mentions potential applications to weakly-supervised learning in Section 5, but it does not provide clear insights into what would be the benefits in terms of representation learning (vs. RGB, pre-trained features, or probability maps).
- Overall, this paper might be more tailored for a Computer Vision venue like CVPR.
Key reason 2: lack of sensitivity / robustness analysis.
- The scribbles are "simulated" using morphological operations on the ground truth (A.2, A.3): does this lead to realistic scribbles? Figure 3 (which is unclear) shows that the "scribbles" might be precise outlines or contours, which are very different than the expected scribbles illustrated in Figure 2. Contours provide much stronger information for segmentation, and are much more likely to effectively leverage the connectivity prior (esp. with the diffusion heuristic), but are they really scribbles / cheap supervision?
- What is the importance of the superpixel coverage by scribbles or missing scribbles or the location of scribbles relative to segment boundaries? What are the impact of realistic deviations from the expected scribble policy that are likely to happen in practice? Measuring sensitivity to different types of noise (by perturbing / dropping out scribbles) seems important to assess the practical usefulness and robustness of the method.
- PASCAL VOC and Cityscapes are small datasets. Experiments on bigger more recent ones like Mapillary Vistas and COCO are becoming the standard protocol in the instance/semantic/panoptic segmentation community. How would this method fare on those much more challenging datasets? What are the benefits of the proposed interactive methods in terms of scalability?
Additional Feedback:
- Fig. 4 is too low resolution / blurry;
- typos: "tarining set", "weekly supervised".
## Update following the rebuttal
Thanks to the authors for their replies. Sadly, my concerns are only answered at a high-level, and the consensus among reviewers is clear. Hence I confirm my rating to reject. I hope the feedback provided above will assist the authors in improving the work or finding a more suitable venue.


Imitation Learning (IL) is a popular paradigm for training agents to achieve complicated goals by leveraging expert behavior, rather than dealing with the hardships of designing a correct reward function. With the environment modeled as a Markov Decision Process (MDP), most of the existing IL algorithms are contingent on the availability of expert demonstrations in the same MDP as the one in which a new imitator policy is to be learned. This is uncharacteristic of many real-life scenarios where discrepancies between the expert and the imitator MDPs are common, especially in the transition dynamics function. Furthermore, obtaining expert actions may be costly or infeasible, making the recent trend towards state-only IL (where expert demonstrations constitute only states or observations) ever so promising. Building on recent adversarial imitation approaches that are motivated by the idea of divergence minimization, we present a new state-only IL algorithm in this paper. It divides the overall optimization objective into two sub-problems by introducing an indirection step, and solves the sub-problems iteratively. We show that our algorithm is particularly effective when there is a transition dynamics mismatch between the expert and imitator MDPs, while the baseline IL methods suffer from performance degradation. To analyze this, we construct several interesting MDPs by modifying the configuration parameters for the MuJoCo locomotion tasks from OpenAI Gym.

Summary:
The manuscript considers the problem of imitation learning when the system dynamics of the agent are different from the dynamics of the expert. The paper proposes Indirect Imitation Learning (I2L), which aims to perform imitation learning with respect to a trajectory buffer that contains some of the previous trajectories of the agent. The trajectory buffer has limited capacity and adds trajectories based on a priority-queue that prefers trajectories that have a similar state distribution to the expert. Similarity is hereby measured by the score of a WGAN-critic trained to approximate the W1-Wasserstein distance between the previous buffer and the expert distribution. By performing imitation learning with respect to a trajectory buffer, state-action trajectories of the agent's MDP are available, which enables I2L to apply AIRL (Fu et al. 2017). By using those trajectories for the transition buffer that have state-marginals close to the expert's trajectory, I2L produces similar behavior compared to the expert. I2L is compared to state-only GAIL, state-action-GAIL and AIRL on four MuJoCo tasks with modified dynamics compared to the expert policy. The experiments show that I2L may learn significantly better policies if the dynamics of agent and the expert do not match.
Decision:
I think that the submission is below borderline in its current state. The main reason for rejecting would be the insufficient justification of some algorithmic choices, improper presentation of the experiments and the description of MaxEnt-IRL, which seems quite wrong in my opinion. However, I think that the work is interesting and sufficiently novel and could be accepted if the mentioned issues were adequately addressed.
Supporting Arguments:
- Novelty / Significance: Imitation learning from state-only observations under dynamic mismatch is an important problem and a promising approach for training robots by non-experts. The idea of performing imitation learning with respect to carefully updated trajectory buffer seems simple and effective. Although self-imitation has been applied in reinforcement learning (references in submission), I am not aware of a similar approach in imitation learning.
- Soundness / Correctness:
1. The description of MaxEnt-IRL seems quite wrong. The paper claims in Section 2.1. that MaxEnt-IRL maximizes the likelihood of the policy by optimizing the advantage function and thus only learns shaped rewards. However, the referenced paper (Ziebart et al. 2008) optimizes the weights of a linear reward function which in general does not correspond to the advantage of the learned policy. Also, the policy is not (only) proportional to the exponentiated advantage but equal to it and the normalizer of the trajectory distribution (Eq. 1) would be therefore 1.
2. I think that the selection of trajectories for the buffer is not sufficiently well motivated. I2L is derived based on a lower bound but it seems that it does not even ensure improvement on that lower bound. It is not fully clear to me how the trajectory buffer is updated (see "Clarity") but it does not seem to ensure that the Wasserstein distance decreases compared to the last iteration. The trajectories are chosen greedily, i.e., without considering the remaining trajectories in the buffer which can be especially problematic for multimodal demonstrations.
- Clarity / Presentation:
The paper is well-written in general and only has few typos. It is not clear to me how exactly the trajectory buffer is updated. The submission merely states that the buffer is a priority-queue structure of fixed lengths, where the trajectory priorities are given by the average state score based on the Wasserstein critic. This description leaves several open questions (see "Questions") and further does not seem well motivated.
- Evaluation:
The presentation of the experimental results seems odd. For the imitation learning experiments (no dynamic mismatch) the (apparently) same runs of I2L are compared to GAIL-S (Table 1) and GAIfO (Table 2) in separate tables. For the experiments under dynamic mismatch, the comparison with GAIL-S are shown with learning curves (Figure 3-5) and shaded error bars, whereas the results of GAIfO are only shown in in terms of final mean performance in Table 2. This presentation may make the impression or hiding learning curves or confidence intervals. Both tables could be removed by adding a single curve to each of the plots in Figure 3-5 and 7. It is also not clear to me, why only the mean of the final performance for AIRL and SA-GAIL is shown in Figure 3-5. Instead, the performance of the expert--which is currently not shown--would be better suited as a baseline level. The paper also does not seem to mention the number of evaluations and the meaning of the error region in the figures. Furthermore, some hyper-parameters, e.g., the architectures for the AIRL reward/value-function networks are not presented.
Questions:
- Please precisely state when the trajectories are removed from the trajectory buffer and the heap of the priority queue. May the same trajectories be used during several iterations?
- Please elaborate: under which assumptions does the update of the trajectory buffer ensure that the W1-distance of the individual trajectories decreases with respect to the expert's trajectories?
Minor comments:
Typo: "upto a constant"
Post-Rebuttal Update:
I increased my rating to weak accept because the authors addressed my main concerns by a) giving additional information on the update of the buffer, b) improving the presentation of the experiments, c) fixing the description of MaxEnt-IRL and d) by showing empirically that both the lower bound increases / Wasserstein-distance decreases.


ABSTRACT Semmelhack et al. (2014) have achieved high classification accuracy in distinguishing swim bouts of zebrafish using a Support Vector Machine (SVM). Convolutional Neural Networks (CNNs) have reached superior performance in various image recognition tasks over SVMs, but their learnt features are not immediately visible. Reaching better transparency helps to build trust in their classifications and makes learned features interpretable to experts. Using a recently developed technique called Deep Taylor Decomposition, we generated heatmaps to highlight input regions of high relevance for predictions. We find that our CNN makes predictions by analyzing the steadiness of the tail's trunk, which markedly differs from the manually extracted features used by Semmelhack et al. (2014). We further uncovered that the network paid attention to experimental artifacts. Removing these artifacts ensured the validity of predictions. After correction, our best CNN beats the SVM by 6.12%, achieving a classification accuracy of 96.32%. Our work thus demonstrates the utility of AI explainability for CNNs.

SUMMARY: explore the use of CNN in a binary task on images of zebrafish
It is important to note that researchers in the field of AI and deep learning are themselves aware of the fallacies of deep learning, and are striving everyday to overcome these themselves. The hype over deep learning has caused certain disdain among a section of the research community over the workings of deep neural networks. This is evident in this paper with the authors calling CNNs "black box" and the learnings of a neural network "cheating". Perhaps the authors are not aware that CNNs are hardly black boxes, their inner workings quite transparent in mathematical terms, which the submitted paper itself explores. Perhaps the authors are also not aware that the fallacies that causes CNNs to overfit on some characteristics in the input data are also present in other machine learning tools such as SVMs. Perhaps the intention of the authors is to bring more relevance to the dangers of spurious correlations, especially when the applications are critical. I hope the community can work together in improving the state of the art while improving transparency and explainability.
BACKGROUND:
Hypothesis: Prey movements in zebrafish are characterized by specific motions, that are triggered by a specific pathway involving an area called AF7.
Validation: Semmelhack et. al. (2014) removed the AF7 neuropil and observed that they failed to respond to prey stimuli
AI: the prey stimuli was a characteristic movement that an SVM was trained to detect.
Good to know that the authors will share their code.
It is not explained why the authors chose to pretrain on ImageNet, since ImageNet does not have any image classes that are comparable to the dataset the authors use. They then proceed to do a hyperparameter sweep to fine-tune on their dataset.
It is also not explained why they chose to average outputs of 500 output nodes to get two outputs, instead of simply replacing the last layer with a 2-neuron layer and finetune, as is general practice.
The level of detail in the training procedure is very helpful to reproduce the setting as well as establish a reference for any future work in this direction. This itself is a notable achievement and a good use of significant research time. Furthermore, the authors conduct one good analysis (DTD) to explain the results of their CNN. The conclusion has many repeated points from previous sections, but it is a good summary.
Given their premise about explainability in machine learning, perhaps more significance was to be given to DTD, and other methods also performed to check if the results coincide with those from DTD.


With the success of deep neural networks, Neural Architecture Search (NAS) as a way of automatic model design has attracted wide attention. As training every child model from scratch is very time-consuming, recent works leverage weight-sharing to speed up the model evaluation procedure. These approaches greatly reduce computation by maintaining a single copy of weights on the super-net and share the weights among every child model. However, weight-sharing has no theoretical guarantee and its impact has not been well studied before. In this paper, we conduct comprehensive experiments to reveal the impact of weight-sharing: (1) The bestperforming models from different runs or even from consecutive epochs within the same run have significant variance; (2) Even with high variance, we can extract valuable information from training the super-net with shared weights; (3) The interference between child models is a main factor that induces high variance; (4) Properly reducing the degree of weight sharing could effectively reduce variance and improve performance.

First of all, I have to state that this is not my area of expertise. So, my review here is an educated guess.
The paper is an empirical study that looks into the effect of weight sharing in neural network architecture search. The basic idea is that by sharing weights among multiple candidate architectures, the training process can be significantly improved.  In the literature, there have been mixed results, either in favor of or against weight sharing. The question this paper aims to address is to determine if weight sharing is justifiable and to what extent.
The primary subject investigated by the authors is to determine if the variance of the ranks generated by different runs of the algorithm are highly correlated with each other (e.g. using Kendall rank correlation score). Then, they compared such results with the ground truth (i.e. every child is trained independently). They found that the ranks generated by weight sharing are indeed highly correlated with each other, but there is much larger variance in the ranks when compared to the ground truth method. To understand why this is non-trivial: (1) on one hand, weight sharing speeds up the training process by providing an initial point close to a local minimum, but (2) the local minimum point may or may not  be good for the new architecture.  Hence, one does not know apriori under what conditions would weight sharing be a good strategy.
The authors also looked into variance of the rank within the same instance by examining how the rank changes with mini-batch epochs. They found that variance is large even within the same instance.
My primary concern is that the paper is entirely empirical with little if any justification of the results. In addition, it is based on a single architecture and a single dataset. This would have been fine if the results were supported with explanation or theoretical justification. Second, the ultimate goal is to improve the prediction accuracy, not the ranking accuracy. These are not necessarily equivalent. For instance, it is possible that the ranks have a high variance simply because many of the candidate architectures have nearly equivalent performance so the order within them becomes nearly random (and unimportant). In fact, I think the results support this conclusion (see for example Figure 10). Third, some of the highlighted observations are trivial. For example, Observation 1, which states that "Two child models have (higher or lower) interference with each other when they share weights. A child model’s validation accuracy highly depends on the child models it is jointly trained with." I think this observation is trivial.
Some other comments:
- I would appreciate it if the authors could explain briefly how "prefix sharing" works so that the paper is self-contained.
- The goal is to help improve the speed of neural architecture search. The authors mention "hints for designing more efficient weight-sharing." Please state those conclusions precisely and clearly. I understand that the authors suggest similarity-based grouping. So, please mention clearly what you recommend in the conclusion section.
==========================
#post rebuttal remarks
Thanks for the response. Figure 10 was a typo from my end and I apologize for it. I actually meant figure 3.
As I said in my review, having an empirical study is acceptable provided that it covers many datasets, not just a single one.


We consider statistical learning problems, when the distribution P of the training observations Z 1 , . . . , Z n differs from the distribution P involved in the risk one seeks to minimize (referred to as the test distribution) but is still defined on the same measurable space as P and dominates it. In the unrealistic case where the likelihood ratio Φ(z) = dP/dP (z) is known, one may straightforwardly extends the Empirical Risk Minimization (ERM) approach to this specific transfer learning setup using the same idea as that behind Importance Sampling, by minimizing a weighted version of the empirical risk functional computed from the 'biased' training data Z i with weights Φ(Z i ). Although the importance function Φ(z) is generally unknown in practice, we show that, in various situations frequently encountered in practice, it takes a simple form and can be directly estimated from the Z i 's and some auxiliary information on the statistical population P. By means of linearization techniques, we then prove that the generalization capacity of the approach aforementioned is preserved when plugging the resulting estimates of the Φ(Z i )'s into the weighted empirical risk. Beyond these theoretical guarantees, numerical results provide strong empirical evidence of the relevance of the approach promoted in this article.

Summary: This paper aims to show that we can estimate a density ratio for using the importance weighted ERM from the given sample and some auxiliary information on the population. Several learning bounds were proven to promote the use of importance weighted ERM.
========================================================
Clarity:
This paper is mathematically concise and understandable overall. Here, I list some comments on the clarity.
1. I found that the word auxiliary information has been used extensively from the very beginning when referring to the estimation of the density ratio. However, there is no explanation what kind of auxiliary information we need to achieve this goal until page 5, where the authors discussed about strata random variable as the additional information (if I didn't make a mistake). I believe there is a better way to introduce the intuition about what kind of auxiliary information is sufficient to make learning possible.
2. For the proposed PU-learning setting (case-control) by du Plessis et al. 2014, the assumption is the marginal density unlabeled data is identical to the test marginal density and the positive data is drawn from the class-conditional probability p(x|y=1). I am not sure if it's appropriate to discuss about it in page 2, where authors want to discuss about the situation where the train stage and test stage have different class probabilities.
3. In page 3, authors suggested that
"it is very common that the fraction of positive instances in the training dataset is significantly lower than the test set (p' < p), supposed to be known here". I have two questions about this.
3.1 Does this mean we suppose to know p', p, or both? I am aware that the appendix discussed about when p is misspecified.
3.2  I am not convinced that it is common that p' < p. It maybe nice to cite some findings or provide more explanation why it is the case.
4. In page 4, authors mentioned few shot learning problem, then describe that it is a scenario where almost no training data with positive labels is available. Is this the same problem setting as the well-known few-shot learning one? In my recognition, few-shot learning is the scenario where we want to learn from small data, e.g., p can be 0.5 but we have a very small number of data but balanced (n_pos=n_neg). Instead of few-shot, I feel it might be better to use the word like "extreme class prior or extreme class probability scenario".
5. In page 3, I'm not sure why authors suddenly focused on binary classification with varying class probabilities. A bit of introduction or motivation would be helpful. As far as I understand this is learning from class-prior shift scenario (or class-prior change), which also has been considered in the literature. Authors may consider citing some work in this line and discuss the difference in the findings of the proposed results and the existing work.
========================================================
Comments:
My impression is the novelty of this paper is modest. It is known that importance weighted ERM is unbiased and consistent to the true risk. I believe there exists theoretical analysis of learning under using WERM, especially in the situation where the weight is importance weight function is known. For Lemma 2 and Corollary 1, it is suggest that p' should not be too small but also the author suggested that p' < p. I would like to know more about the setting the authors described here, e.g., what is the example of the practical p' and p.
1. Eq. (11) is identical to the proposed unbiased risk estimator of PU-learning in du Plessis et al. (NeurIPS2014). It would be better to clarify that they are equivalent (Eq. (3) of PU-learning in du Plessis et al. (NeurIPS2014)). They also provided a generalization error bound and the analysis when p is misspecified. More theoretical analysis of this empirical risk estimator for case-control PU learning (e.g., estimation error bound) can also be found in the following paper:
Niu et al. Theoretical Comparisons of Positive-Unlabeled Learning against Positive-Negative Learning, NeurIPS2016.
2. How many trials were run in the experiments? It would be nice to see the standard error not only the mean result. It is known that importance weighting method can have high variance and it might be expected that WERM may have high variance yet have better performance. It would be helpful to explain how to read the table, e.g., what is No Bias, top-5 error. Why half of the table are in gray?
Although the paper is well-written overall. I found that it is difficult to quantify a novelty of this paper. I believe the goal, as suggested by page 2, is to "set theoretical grounds for the application of ideas behind weighted ERM". As the author suggested, this approach has been studied quite extensively both theoretically and experimentally. It would be helpful to explain what is new and the relationship of the proposed methods or bounds with the existing work to highlight the novelty of this paper.
For these reasons, I vote a weak reject for this paper.
========================================================
Potential typos:
1. There are "du Plessis et al." and "Du Plessis et al." in this paper. This indicates the same person and it should be better to use only one convention (I think du Plessis is preferable).
=========================================================
After Rebuttal:
Thanks for the reader for clarifying my several questions. I have read the rebuttal.
However, I feel that in the current form, I would like to stay with the same evaluation. The clarification of the difference between theoretical results is definitely crucial to highlight the novelty of the paper. I would like to add more comments on the PU learning part. I hope the authors find the comments useful.
1. du Plessis et al. (ICML 2015) "Convex formulation for learning from positive and unlabeled data", which was already cited in the paper, suggested that if we replace the 0-1 loss to a loss that does not have the symmetric property (e.g., logistic, squared), the form of the unbiased estimator can be different from Eq. (11) in this submitted paper (please see the paper for more details).

2. Although we obtain an unbiased risk estimator by the WERM-like method, in deep learning, minimizing such a risk may lead to overfitting, as we can see from Eq. (11) that although it is a cost-sensitive risk, it still treats all unlabeled data as negative. If we have a complex enough model, to minimize the risk, a classifier may classify all unlabeled data to negative, which undoubtedly leads to overfitting. This is discussed in Kiryo et al. (NeurIPS 2017), which also has been already cited in the submitted paper too.
Next, I would like to add more comments on the experiment.
For the experiments, I appreciate the authors' effort to do experiments on such a big dataset. In that case, it may be nice to also include an experiment on a smaller dataset, e.g., MNIST, which I believe this has already been conducted but it was in the appendix, to the main body of the paper as well to strengthen the experimental results in the paper.
I think the writing in the experiment section can be improved. For example, I don't see the first paragraph, which has lots of texts, contains much information. Also, instead of suggesting a reader to see Figure 1 for comparison, we may use more space to interpret the result. If I didn't miss it, Figure 2 and Figure 3 were never explained or referred to in the main body. In that case, we may consider removing these figures and adding the result on MNIST dataset.


Most algorithms for representation learning and link prediction in relational data have been designed for static data. However, the data they are applied to usually evolves with time, such as friend graphs in social networks or user interactions with items in recommender systems. This is also the case for knowledge bases, which contain facts such as (US, has president, B. Obama, [2009][2010][2011][2012][2013][2014][2015][2016][2017]) that are valid only at certain points in time. For the problem of link prediction under temporal constraints, i.e., answering queries such as (US, has president, ?, 2012), we propose a solution inspired by the canonical decomposition of tensors of order 4. We introduce new regularization schemes and present an extension of ComplEx (Trouillon et al., 2016) that achieves state-of-the-art performance. Additionally, we propose a new dataset for knowledge base completion constructed from Wikidata, larger than previous benchmarks by an order of magnitude, as a new reference for evaluating temporal and non-temporal link prediction methods.

The work in this paper is focused on the task of knowledge base completion, dealing specifically with temporal relations, which is quite important in practice, and also not as well studied in literature. Specifically, the authors have 3 main contributions in this paper:
1. They present an order 4 tensor factorization for dealing with temporal data, which is a nice extension of the work in Lacroix 2018. The authors introduce different forms of regularization to extend to the order 4 tensors. Inspired by previous work, they produce different regularization strategies for order 4 tensor by unfolding the modes to reduce it ot an order 3 formulation. They also describe a regularization term for smoothing the temporal embeddings.
3. Finally, the authors mine wikidata for temporal relations and contribute a dataset based on wikidata that is much larger than existing datasets for this task.
Overall, I think this paper is  interesting as it provides an incremental extension of ComplEx to the temporal case, and the experiments to support the formulation show improvements in MRR. However, the experiments around standard benchmarks as well as the data produced by the authors do not always support the hypothesis that modeling temporal dimension in the proposed formulation is a win for the KB completion task.  For example, the use of auxiliary losses  for enforcing temporal constraints makes the overall performance worse. This is mentioned in a section, but I think it deserves a more thorough explanation. Also, there is no mention about statistical significance of the results, and so it is hard to judge the claim of these being SOTA as made by the authors. For example, on the wikidata dataset produced, the ComplEx model outperforms the proposed models.
Strengths:
1. The work in this paper is quite well motivated and the modeling formulation is clear and easy to follow. The authors did a great job in citing relevant work and walking through the model formulation as well as the various regularization terms introduced.
2. The authors compare their work to the previous SOTA - the ComplEx models for multiple datasets, including their own released datasets. They present a clear set of experiments to demonstrate the effectiveness of their approach both on non-temporal and temporal relations.
3. There is a dataset being released and also code, which should aid in reproducibility of the results (though I have not tested the code).
Areas to be addressed:
1. The work seems to assume that the time is discretized by year. However, it is unclear how one would deal with a KB where the temporal relations can change on the order of weeks (example popular movies). For that matter, how would the model be modified to deal with heterogenous time scales in the evolution of the relations in the KB? Can the authors add some clarification as well as explanations for how this would be addressed in this formulation?
2. While there is a section devoted to different regularization, from Table 3, the impact of choosing the right regularizer seems minimal in terms of MRR. Also, it seems that the results in Table 3 are comparable to the “ranks multiplied by 10” setting in Table 2. What is the reason for this choice?
3. For Table 4, why is there a performance difference between static and non-static relations? It would help if the authors could provide some more error analyses to dive into this performance difference - is it just data imbalance or inherent difficulty in the task for temporal relations.
4. Section 6 talks about enforcing another auxiliary loss, however, these results are not part of the table. I would urge the authors to add this to Table 4. Also, the loss in MRR mentioned is it the average loss or does enforcing this auxiliary loss also influence performance on T-MRR as well? If so, might it be that the auxiliary loss is too strong and might need to be penalized? Did the authors try penalizing the auxiliary loss? Finally, the graph in Figure 2 is hard to follow when printed in black and white. What would the plot look like in comparison to a model that does not enforce the auxiliary function for the same example? I would recommend re-working Section 6 to provide some more details about the performance of the models both across the various datasets as well as error analyses of a few examples compared across TNTComplex, TNTComplex + auxiliary loss + Baseline Complex. Having some representative examples would make it easy to understand where these models differ in their performance.
5. In the results, can the authors include metrics like filtered MRR as well as Hits@10, similar to the one suggested by Bordes 2013? This would make it easier to compare against previous literature results which all seem to be reporting on these metrics.


Backdoor attacks aim to manipulate a subset of training data by injecting adversarial triggers such that machine learning models trained on the tampered dataset will make arbitrarily (targeted) incorrect prediction on the testset with the same trigger embedded. While federated learning (FL) is capable of aggregating information provided by different parties for training a better model, its distributed learning methodology and inherently heterogeneous data distribution across parties may bring new vulnerabilities. In addition to recent centralized backdoor attacks on FL where each party embeds the same global trigger during training, we propose the distributed backdoor attack (DBA) -a novel threat assessment framework developed by fully exploiting the distributed nature of FL. DBA decomposes a global trigger pattern into separate local patterns and embed them into the training set of different adversarial parties respectively. Compared to standard centralized backdoors, we show that DBA is substantially more persistent and stealthy against FL on diverse datasets such as finance and image data. We conduct extensive experiments to show that the attack success rate of DBA is significantly higher than centralized backdoors under different settings. Moreover, we find that distributed attacks are indeed more insidious, as DBA can evade two state-of-the-art robust FL algorithms against centralized backdoors. We also provide explanations for the effectiveness of DBA via feature visual interpretation and feature importance ranking. To further explore the properties of DBA, we test the attack performance by varying different trigger factors, including local trigger variations (size, gap, and location), scaling factor in FL, data distribution, and poison ratio and interval. Our proposed DBA and thorough evaluation results shed lights on characterizing the robustness of FL.

This paper studies backdoor attacks under federated learning setting. To inject a certain backdoor pattern, existing work generate poisoning samples by blending the same pattern with different input samples. Even for federated learning where the adversary can control multiple parties, such as [1], all parties still use the same global backdoor pattern to generate poisoning samples locally. On the contrary, in this work, they decompose the global pattern into several small local patterns, and each adversarial party only uses a local pattern to generate poisoning samples. In their evaluation, they show that the backdoor attacks generated in this way are more effective, resilient to benign model parameter updates, and also survive better against existing defense algorithms against attacks in federated learning settings.
I think the topic studied in this paper is very important and meaningful, and I am convinced that by decomposing a global pattern into several smaller local pieces, the model parameter updates computed by each party should be more similar to benign updates and thus can better bypass the defense algorithms. Meanwhile, the evaluation is pretty comprehensive and it is good to see that the conducted backdoor attacks are effective. However, when there is no defense deployed in the training process, it is not intuitive to see why the proposed attack is more effective and persistent than the centralized attack, given that a smaller trigger usually results in a worse attack performance. Thus, I would like to see more possible explanation on it. Specifically, I have the following questions for clarification:
1. For the evaluation of DBA, I assume that there are 4 adversarial parties, controlling each of the 4 local triggers. When using centralized attacks, are there still 4 adversarial parties, although they share the same global trigger, or if there is only 1 adversarial party?
2. To evaluate A-S setting, I understand that it may be tricky to enable a fair comparison between the centralized attack and DBA. However, one explanation of why DBA is more persistent in this case is because the adversarial parameter updates happen 4x times compared to the centralized attack. Therefore, another baseline to check is to conduct centralized attacks with the same number of times as DBA, but each update includes 1/4 number of poisoning samples, so that the total number of poisoning samples included to compute the gradient update still stays the same.
3. Can the authors show if the decomposition is also useful for trigger patterns that are not necessarily regular shapes? For backdoor attacks, a line of work studies physical triggers, e.g., glasses in [2]. It is not natural to decompose such kind of patterns into several smaller pieces, unless the performance is significantly boosted.
4. Can the authors show concrete examples on how the attacks are generated? The details are especially unclear on LOAN. Specifically, which features are perturbed, what are the values assigned as the trigger, and what is the corresponding target label?
[1]  Bagdasaryan et al., How to backdoor federated learning.
[2] Chen et al., Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning.
------------
Post-rebuttal comments
I appreciate the authors' great effort to address my concerns! I think the evaluation in the current version of the paper is pretty comprehensive and provides a valuable study, and I am happy to raise my score accordingly.
-------------


This paper proposes the use of spectral element methods (Canuto et al., 1988) for fast and accurate training of Neural Ordinary Differential Equations (ODE-Nets; Chen et al., 2018) for system identification. This is achieved by expressing their dynamics as a truncated series of Legendre polynomials. The series coefficients, as well as the network weights, are computed by minimizing the weighted sum of the loss function and the violation of the ODE-Net dynamics. The problem is solved by coordinate descent that alternately minimizes, with respect to the coefficients and the weights, two unconstrained sub-problems using standard backpropagation and gradient methods. The resulting optimization scheme is fully time-parallel and results in a low memory footprint. Experimental comparison to standard methods, such as backpropagation through explicit solvers and the adjoint technique (Chen et al., 2018), on training surrogate models of small and medium-scale dynamical systems shows that it is at least one order of magnitude faster at reaching a comparable value of the loss function. The corresponding testing MSE is one order of magnitude smaller as well, suggesting generalization capabilities increase.

This work extends prior work on Neural ODEs.  From what I understand, the Neural ODE approach builds off the idea of representing the sequence of transformations of a hidden state (in residual nets, RNNs, etc.) as an ODE parameterized by a neural network.   In the original paper, the network is optimized via gradients calculated by the adjoint sensitivity method.  This paper puts forth the following contributions: a compact representation of the state transition function as a combination of Legendre polynomials, and an optimization scheme whose error is tied to the polynomial order and whose structure lends itself easily to parallelization.  The authors also demonstrate their model on an experiment on planar vehicle dynamics, in which their model is shown to have improved predictive quality and efficiency.
I am inclined to accept this paper, due to its various contributions on speed and performance, with the caveat for some clarifications on how the experiments were conducted and compared.  Given these clarifications in an author response, I would be willing to increase the score.
Pros of the paper:
	1) Trajectory predictions on the two planar vehicle dynamics experiments was impressive.
	2) The proposed representation of the state transition dynamics is indeed more memory-efficient, and its approximation error is modulatable by the hyperparameter order \p.
	3) Speedup due to parallelization is substantial.
Cons of the paper:
	1) The experiments did not display a proper comparison against the hybrid method mentioned in Section 1.  The experiments also did not compare against adjoint methods in the multi-agent example, or in the low-data regime for the single-agent example.  Instead, the experiments mostly highlighted the problems with direct backpropagation through the ODE solver, which is already well-known to have issues in robustness and stability.  While it is nice to have empirical results that showcase this, a more comprehensive comparison against current adjoint methods would be more interesting, especially in the multi-agent example.
	2) It slightly detracts from the cleanliness of the story that we must first create an initial trajectory, before performing our coordinate descent.
Questions and Points of Confusion:
	1) What intuits the choices of the Legendre polynomial as your set of basis functions, and the Gauss-Lobatto scheme to select collocation points, instead of alternative candidates?
	2) In Section 5, it was mentioned that "100 equally-spaced points produce a comparable result" to the Gauss-Lobatto quadrature points.  Furthermore, it was mentioned that these evenly-spaced collocation points were used for experiments - was this the case for all experiments?  If so, then what purpose does Gauss-Lobatto play in the paper?
	3) In Figure 1, were the DoPr5 and Euler plots shown for the backpropagation or adjoint method?  If it was the backpropagation method, the plots for the adjoint method would be highly interesting to show.
	4) From what I understand from Section 4, when we perform step 0 and step 2 to update the trajectory, we simply optimize the coefficients \x_i and \u_i directly to minimize the current objective as both \x(t) and \u(t) are represented as a combination of Legendre polynomials.  However, in Equation 8, for the planar vehicle dynamics, \u is deterministically generated from the current states and network weights.  It makes sense to add structure to \u, as we need a way to make sure the inputs can indeed generate the trajectory of \x.  Does this mean the spectral method is not performed for \u as was detailed in Equation 5?
	5) I am confused about how the gray-box models are built in Section 5.  It states that "For \f_J, sin(\phi) and cos(\phi) are used as input features, where \phi is the vehicle orientation."  Does that mean that only \phi from \eta is passed in as an input, both sin(\phi) and cos(\phi) are passed in as inputs, or the entire current \eta is passed in as input?  And is the output a new \eta, which is then structured into the 3x3 matrix \J(\eta) in Appendix A?  Or does it simply output the matrix directly for the given value of \phi.  I am confused because it is written that \J(\eta) is equal to \f_J(\eta;\theta_J), but then in the appendix it is written that \J(\eta) is equal to the matrix - so where is the network?  This confusion extends to the other gray-box models \C(v) and \d(v).
	6) It is written in Equation 3 that the residual also takes in the input \u(t).  However, in the experiments, namely Equation 8, it does not appear like \u is used to calculate the residual at all.
	7) What motivated the use of the planar vehicle dynamics experiment to showcase your model?  Were there other baselines or benchmarks you considered or attempted?


The use of adequate feature representations is essential for achieving high performance in high-level human cognitive tasks in computational modeling. Recent developments in deep convolutional and recurrent neural networks architectures enable learning powerful feature representations from both images and natural language text. Besides, other types of networks such as Relational Networks (RN) can learn relations between objects and Generative Adversarial Networks (GAN) have shown to generate realistic images. In this paper, we combine these four techniques to acquire a shared feature representation of the relation between objects in an input image and an object manipulation action description in the form of human language encodings to generate an image that shows the resulting end-effect the action would have on a computer-generated scene. The system is trained and evaluated on a simulated dataset and experimentally used on real-world photos.

1. The paper aims to train a model to move objects in an image using language. For instance, an image with a red cube and blue ball needs to be turned into an image  with a red cube and red ball if asked to "replace the red cube with a blue ball". The task itself is interesting as it aims to modify system behavior through language.
The approach the authors  take is to encode an image with a CNN, encode the sentence with an RNN and use both representations to reconstruct  the frame (via a relational network and decoder) that solves this task. This process as described was already done in (Santoro 2017). The idea of using a CNN feature map and LSTM embedding to solve spatial reasoning tasks is not new.
The main contribution is to add a discriminative loss to turn the problem into a "is this solution correct or not." This is interesting but does not perform much better than the baseline of not using the GAN loss (as suggested by the results in Table 2). This suggests that the GAN term is not adding as much value as the authors claim.
2. Reject
- Reason 1: The results in Table 2 show that the GAN does slightly better (0.0134 vs 0.0144) in RMSE against the non-GAN version. This improvement does not seem statistically significant enough to warrant the added GAN complexity.
- Reason 2: Other baselines need to be considered, AE, VAE or other variations.
- Reason 3: No ablations on the impact of the parameters to eq 1.
3. To improve the paper I suggest adding other baselines such as VAE, AE. In addition, consider using more negative samples instead of the single negative image.


Runtime and scalability of large neural networks can be significantly affected by the placement of operations in their dataflow graphs on suitable devices. With increasingly complex neural network architectures and heterogeneous device characteristics, finding a reasonable placement is extremely challenging even for domain experts. Most existing automated device placement approaches are impractical due to the significant amount of compute required and their inability to generalize to new, previously held-out graphs. To address both limitations, we propose an efficient end-to-end method based on a scalable sequential attention mechanism over a graph neural network that is transferable to new graphs. On a diverse set of representative deep learning models, including Inception-v3, AmoebaNet, Transformer-XL, and WaveNet, our method on average achieves 16% improvement over human experts and 9.2% improvement over the prior art with 15× faster convergence. To further reduce the computation cost, we pre-train the policy network on a set of dataflow graphs and use a superposition network to fine-tune it on each individual graph, achieving state-of-the-art performance on large hold-out graphs with over 50k nodes, such as an 8-layer GNMT.

GDP: Generalized Device Placement for Dataflow Graphs
This paper presents a method to assign the individual operations making up the dataflow graph of a deep neural network to a set of connected devices on which they should be executed, with the objective of maximizing runtime performance of inference. The paper’s proposed approach relies on graph neural networks to produce embeddings of the dataflow graph nodes that are claimed to be transferable to unseen graphs. These embeddings serve as input to a transformer-based sequence model that outputs the final assignment. The end-to-end model is trained with a reinforcement learning criterion minimizing the expected placement runtime. Experimental comparisons against alternative placement methods (including human expert) are presented for several large neural networks of different architectures.
Overall, the paper is well written, easy to follow, and addresses an important concern is scaling up neural networks to large model sizes. The proposed approach combining graph neural networks with transformer-based placement network appears, at first glance, novel.
My main reservations with the paper is that it lacks many details in several key sections, preventing a full appreciation of the contributions, and making reproducibility of results impossible to contemplate. In particular:
1. How the placement network is designed and used (section 3.2) is completely lacking.
2. The details of how the training and testing datasets are obtained are also lacking. In particular, the specific model architectures for the likes of RNNLM, including hyperparameter choices, the runtime data fed to those models, etc. (This can be given in appendix). It is also not clear if the models listed in Table 1 are singular models with fixed values of hyperparameters, or if they correspond to distributions over models (e.g. with differences in hyperparameters).
3. How PPO is used is glossed over: for instance, is the proposed placement of a given model tried « live » in the inner loop of PPO to generate the reward corresponding to its runtime? What runtime data is fed to the model in order to do this? Since this would be a somewhat unusual setup, an illustration of the overall training loop would certainly solidify understanding. Moreover, a few sentences or equations explaining how PPO is used in this context would also help. Is training with PPO imperative for getting good model performance? Have other methods been tried?
In section 3.3, the parameter superposition mechanism appears to perform a kind of meta-learning. Explicit connections to such should be made, such as « Tadam: Task dependent adaptive metric for improved few-shot learning » (NeurIPS 2018).
Moreover, at the end of section 4.4, the paper claims to « report superhuman results on 8-layer GNMT ». Unless this reviewer misunderstands, the results in Table 1 (wherein GDP-one clocks in at 0.649 for 8-layer GNMT, versus 0.610 for Human Placement) would contradict this claim. This should be clarified.
Given these reservations, in spite of the potential of the proposed approach, the paper appears in its current form too immature to recommend acceptance at ICLR.