Pruning and quantization are typical approaches to reduce the computational cost of convolutional neural network (CNN) inference. Although the idea of combining both approaches seems natural, it is surprisingly difficult to determine the effects of the combination without measuring performance on the specific hardware that the user will use. This is because the benefits of pruning and quantization strongly depend on the hardware architecture where the model is executed. For example, a CPU-like architecture with no parallelization may fully exploit the reduction of computations by unstructured pruning to improve speed, but a GPUlike massive parallel architecture would not. Further, there have been emerging proposals of novel hardware architectures, such as those supporting variable bitprecision quantization. From an engineering viewpoint, optimization for each hardware architecture is useful and important in practice, but this is in essence a brute-force approach. Therefore, in this paper, we first propose a hardwareagnostic metric for measuring computational costs. Using the proposed metric, we demonstrate that Pareto-optimal performance, where the best accuracy is obtained at a given computational cost, is achieved when a slim model with fewer parameters is moderately quantized rather than a fat model with a huge number of parameters is quantized to extremely low bit precision, such as binary or ternary. Furthermore, we empirically find a possible quantitative relation between the proposed metric and the signal-to-noise ratio during stochastic gradient descent (SGD) training, by which information obtained during SGD training provides an optimal policy for quantization and pruning. We show the Pareto frontier is improved by 4× in a post-training quantization scenario based on these findings. These findings not only improve the Pareto frontier for accuracy versus computational cost, but also provide new insights into deep neural networks.

The paper proposes a new metric to evaluate both the amount of pruning and quantization. This metric is agnostic to the hardware architecture and is simply obtained by computing the Frobenius norm of some point-wise transformation of the quantized weights. They first show empirically that this Evaluation metric is correlated with the validation accuracy. Then use this metric to provide some general rules for pruning/quantizing to preserve the highest validation accuracy. Finally, they derive a strategy to perform pruning by monitory the signal to noise ratio during training and show experimentally that such method performs better than competing ones.
Pros: - Extensive experiments were performed to test the methods, and the results seem promising.
Cons: - The paper is not very clear, and the structure is somehow confusing.
- It is not easy at first to understand the experimental setup and requires to make a lot of guesses in my opinion.
- The paper didn't motivate properly the use of a hardware-agnostic metric in the context of the quantization and pruning. Isn't the ultimate goal of pruning/quantization is to optimize the run time/energy consumption of the specific device with the least compromise on the accuracy?
I feel that the paper currently jumps between very different ideas:
	- Evolution of the proposed metric during training: 2.3 and 2.5. While the 2.3, the take-home message is relatively clear:  the ESN is correlated with the validation accuracy, I don't fully get the point of section 2.5: It suggests that the optimizer does some sort of pruning just by choosing a higher learning rate.
	- Finding an optimal strategy for pruning/quantizing a network: 2.4 and 2.6. Those two sections are relatively clear, although I have some questions about the experiments.
	- Developing a new strategy based on the proposed metric to quantize and prune a network in a Pareto optimal sense: This is briefly and not very well explained in section 2.7, which sends back to 2.3, but it is hard to understand how it is exactly done. It seems that section 3 provides some empirical evidence supporting this strategy, but the description of the method is hidden in the experimental details.
Some questions:
- In figure 3, the blue dots represent validation vs ESN at each training iteration? What about the red plot, is it obtained by quantization of the parameters at different stages of training, or is it using the final parameters? Which equation was used to compute the red curve (2) or (4)?  How much quantization was performed? If the quantization was chosen to match the level of noise then it seems natural to expect such behavior in figure 3.
- In figure 4, how much pruning was performed for each network and was it the same quantization? In other words how each point in the plot was obtained?  The authors come to the conclusion that one should 'prune until the limit of the desired accuracy and then quantize', but it is hard for me to reach the same conclusions as I don't see the separate effect of pruning and quantization in those figures. Or maybe pruning is implicitly done by choosing a small network? In this case, it makes more sense, but still, some clarifications are needed.
- Which equation for the ESN was used to produce figure 5? Equation (2) or (4)?
- What is the Pareto frontier? I think it is worth first introducing this concept and describing more precisely how those curves are obtained. For someone who is not very familiar with these concepts, which is my case, it makes the reading very hard.
- How was the number of pruned filters computed in figure 5 (right)? I don't expect the solutions to be sparse during training, especially that no sparsity constraint was imposed, or was it?

-------------------------------------------------------------------------------
Revision:
Thank you for all the clarifications and the effort to make provide a clearer version of the paper.
Regarding section 2.7: ESNa FOR QUANTIZATION: Would it make sense to include the paragraph 2.7 at the end of 2.3, since it related to it and doesn't seem to require any of the intermediate subsections.
response to Comment 3: Unfortunately, I'm not convinced by the explanation about the effect of the lr on sparsity. The decay coefficient controls the saparsity indeed, but not the lr. That is because unlike the lr, the decay coefficient defines the cost functions to be optimized:  L+dc ||W||^2, while the lr corresponds simply to the discretization of some gradient flow.   For instance, in a deterministic and convex setting, the solution that is obtained would be the same, regardless of the chosen lr  ( provided the lr is smal enough so that the algorithm converges) see for instance [1].  In a non-convex and stochastic setting do the authors have a particular reference in mind? I'm not aware of such behavior. I would expect a similar sparsity if dc is kept fixed and only lr changed. Is it likely that with smaller lr, the algorithm just didn't have time to converge? This would explain why the obtained solutions were less sparse.
response to answer 5:  it is indeed well known that L1 norm induces sparsity, however l2 doesn't, it just encourages the weights to be smaller. In the optimization litterature sparsity of x% means x% of the parameters  are exactly 0. This is achieved with l1 norm, however l2 norm would only enforce that the coefficients are small but not necessarily 0 (see [1])
[1]:  ROBERT TIBSHIRANI, Regression Shrinkage and Selection via the Lasso.
Although the paper improved in terms of clarifications and experimental details, I still think it will benefit from additional work on careful interpretation of the results.


Many cooperative multiagent reinforcement learning environments provide agents with a sparse team-based reward, as well as a dense agent-specific reward that incentivizes learning basic skills. Training policies solely on the team-based reward is often difficult due to its sparsity. Also, relying solely on the agent-specific reward is sub-optimal because it usually does not capture the team coordination objective. A common approach is to use reward shaping to construct a proxy reward by combining the individual rewards. However, this requires manual tuning for each environment. We introduce Multiagent Evolutionary Reinforcement Learning (MERL), a split-level training platform that handles the two objectives separately through two optimization processes. An evolutionary algorithm maximizes the sparse team-based objective through neuroevolution on a population of teams. Concurrently, a gradient-based optimizer trains policies to only maximize the dense agent-specific rewards. The gradient-based policies are periodically added to the evolutionary population as a way of information transfer between the two optimization processes. This enables the evolutionary algorithm to use skills learned via the agent-specific rewards toward optimizing the global objective. Results demonstrate that MERL significantly outperforms state-of-the-art methods, such as MADDPG, on a number of difficult coordination benchmarks.

This paper proposes an integration of neuroevolution and gradient-based learning for reinforcement learning applications. The evolutionary algorithm focuses on sparse reward and multiagent/team optimization, while the gradient-based learning is used to inject selectively improved genotypes in the population.
This work addresses a very hot topic, i.e. the integration of NE and DRL, and the proposed method offers the positive side of both without introducing major downsides. The presented results come from a relatively simple but useful multiagent benchmark which has broad adoption. The paper is well written, presents several contributions that can be extended and ported to other work, and the results are statistically significant.
There is one notable piece missing which forces me to bridle my enthusiasm: a discussion of the genotype and of its interpretation into the network phenotype. The form taken by the actual agent is not explicitly stated; following the adoption of TD3 I would expect a policy and two critics, for a grand total of three neural networks, but this remains unverified. And if each agent is composed of three neural networks, and each individual represents a team, does this mean that each genotype is a concatenation of three (flattened) weight matrices per each agent in the team? What is the actual genotype size? It sounds huge, I would expect to be at least several hundred weights; but then this would clash with the proposed minuscule population size of 10 (recent deep neuroevolution work from Uber uses populations THREE orders of magnitude larger). Has the population size been proportionated to the genotype dimensionality? Would it be possible to reference the widely adopted defaults of industry standard CMA-ES? Speaking of algorithms, where is the chosen EA implementation discussed? The overview seems to describe a textbook genetic algorithm, but that has been overtaken as state-of-the-art since decades, constituting a poor match for TD3.
Omitting such a chapter severely limits not only the reproducibility of the work but its full understanding. For example, does the EA have sufficient population size to contribute significantly to the process, or is it just performing as a fancy version of Random Weight Guessing? Could you actually quickly run RWG with direct policy search (rather than random action selection) to establish the effective complexity of the task? My final rating after rebuttal will vary wildly depending on the ability to cover such an important piece of information.
A few minor points, because I think that the paper appearance deserves to match the quality of the content:
- The images are consistently too small and hard to read. I understand the need to fit in the page limit by the deadline, but for the camera ready version it will be necessary to trim the text and rescale all images.
- The text is well written but often slowing down the pace for no added value, such as by dedicating a whole page to discussing a series of previously published environments.
- The hyperparameters of the evolutionary algorithm look completely unoptimized. I would expect a definite improvement in performance with minimal tuning.
- The "standard neuroevolutionary algorithm" from 2006 presented as baseline has not been state-of-the-art for over a decade. I would understand its usage as a baseline if that is indeed the underlying evolutionary setup, but otherwise I see no use for such a baseline.
-----------------------------------------------------------------------------------------------
# Update following the rebuttal phase
-----------------------------------------------------------------------------------------------
Thank you for your work and for the extended experimentation. I am confident the quality of the work is overall increased.
The core research question behind my original doubt however remains unaddressed: does the EC part of the algorithm sensibly support the gradient-descent part, or is the algorithm basically behaving as a (noisy) multi-agent TD3?
Such a contribution by itself would be undoubtedly important. Submitting it as a principled unification of EC and DL however would be more than a simple misnomer: it could mislead further research in what is an extremely promising area.
The scientific approach to clarify this point would be to design an experiment showcasing the performance of MARL using a range of sensible population sizes. To understand what "sensible" means in this context, I refer to a classic:
http://www.cmap.polytechnique.fr/~nikolaus.hansen/cec2005ipopcmaes.pdf
A lower bound for the population size with simple / unimodal fitness functions would be $4+floor(3*log(10'000)) = 31$. With such a complex, multimodal fitness though, no contribution from the EA can be expected (based on common practice in the EC field) without at least doubling or tripling that number. The upper bound does not need to be as high as with the recent Uber AI work (10k), but certainly showing the performance with a population of a few hundreds would be the minimum necessary to support your claim. A population size of 10 represents a proper lower bound for a genotype of up to 10 parameters; it is by no means within a reasonable range with your dimensionality of 10'000 parameters, and no researcher with experience in EC would expect anything but noise from such results -- with non-decreasing performance uniquely due to elitism.
The new runs in Appendice C only vary the population size for the ES algorithm, proposed as a baseline. No performance of MARL using a sensible population size is presented.
The fundamental claim is thereby unsustainable by current results. The idea is extremely intriguing and very promising, easily leading to supportive enthusiasm; it is my personal belief however that accepting this work in such a premature stage (and with an incorrect claim) could stunt further research in this direction.
[By the way, the reference Python CMA-ES implementation runs with tens of thousands of parameters and a population size of 60 in a few seconds per generation on a recent laptop: the claim of performance limitations as an excuse for not investigating a core claim suggests that more work would be better invested prior to acceptance.]


State-of-the-art machine learning methods exhibit limited compositional generalization. At the same time, there is a lack of realistic benchmarks that comprehensively measure this ability, which makes it challenging to find and evaluate improvements. We introduce a novel method to systematically construct such benchmarks by maximizing compound divergence while guaranteeing a small atom divergence between train and test sets, and we quantitatively compare this method to other approaches for creating compositional generalization benchmarks. We present a large and realistic natural language question answering dataset that is constructed according to this method, and we use it to analyze the compositional generalization ability of three machine learning architectures. We find that they fail to generalize compositionally and that there is a surprisingly strong negative correlation between compound divergence and accuracy. We also demonstrate how our method can be used to create new compositionality benchmarks on top of the existing SCAN dataset, which confirms these findings.

This paper first introduces a method for quantifying to what extent a dataset split exhibits compound (or, alternatively, atom) divergence, where in particular atoms refer to basic structures used by examples in the datasets, and compounds result from compositional rule application to these atoms. The paper then proposes to evaluate learners on datasets with maximal compound divergence (but minimal atom divergence) between the train and test portions, as a way of testing whether a model exhibits compositional generalization, and suggests a greedy algorithm for forming datasets with this property. In particular, the authors introduce a large automatically generated semantic parsing dataset, which allows for the construction of datasets with these train/test split divergence properties. Finally, the authors evaluate three sequence-to-sequence style semantic parsers on the constructed datasets, and they find that they all generalize very poorly on datasets with maximal compound divergence, and that furthermore the compound divergence appears to be anticorrelated with accuracy.
This is an interesting and ambitious paper tackling an important problem. It is worth noting that the claim that it is the compound divergence that controls the difficulty of generalization (rather than something else, like length) is a substantive one, and the authors do provide evidence of this. At the same time, I think the authors could possibly do more to show that the trend in the plots in Figure 2 can't be explained by something else: for example, the authors could show that the length ratios remain constant as the compound divergence is varied.  I think it is also not necessarily clear how easily the notion of differing compound distributions generalizes to other types of tasks.
Presentation-wise, much of the paper is clear and well written, though I think the discussion of weighted frequency distributions of compounds (top of page 3) could be clarified further, and in particular an example subgraph of a rule application DAG should be highlighted here.


Explaining a deep learning model can help users understand its behavior and allow researchers to discern its shortcomings. Recent work has primarily focused on explaining models for tasks like image classification or visual question answering. In this paper, we introduce an explanation approach for image similarity models, where a model's output is a score measuring the similarity of two inputs rather than a classification. In this task, an explanation depends on both of the input images, so standard methods do not apply. We propose an explanation method that pairs a saliency map identifying important image regions with an attribute that best explains the match. We find that our explanations provide additional information not typically captured by saliency maps alone, and can also improve performance on the classic task of attribute recognition. Our approach's ability to generalize is demonstrated on two datasets from diverse domains, Polyvore Outfits and Animals with Attributes 2.

Overview/Contribution:
====================
The paper proposes an explanation mechanism that pairs the typical saliency map regions together with attributes for similarity matching deep neural networks. The authors tested their methods on two datasets, i.e. Polyvore Outfits (a clothing attributes dataset) and Animals with Attributes 2 (a dataset of attributes for animals).
Overall, the paper has merit to be accepted to the conference with the following strengths and weaknesses. I suggest to the authors to address the weaknesses pointed out to make the paper more stronger, especially adding few more attributes datasets such as person attributes datasets as noted below in the weakness section.
Strength:
========
- The paper is written clearly and is easy to understand. I have seen the additional results and visual comparisons in the supplemental material and it was useful, albeit a bit longer.
- Explanations have the potential to make decisions made by a deep neural model transparent to end users among other benefits especially for sensitive applications such as healthcare and security. Explaining decisions made by similarity matching models has many applications including person attribute recognition and person re-identification for surveillance scenarios [1]. So, in this respect, this paper is relevant to the target audience.
- There is a bit of confusion between explanation and interpretation of decisions made by deep neural network models in the explainable AI literature and in most cases the two are used interchangeably. Hence, saliency maps are considered as explanation on their own by many. Combining saliency map based interpretations together with higher level concepts such as attributes has the potential to generate more realistic explanations of the decisions. The authors made this point at the second paragraph of the introduction.
- Fig. 1 (b) also is a clear example of the kind of explanations generated using a template with the key attribute in question accompanied by the visual saliency map interpretation.
- Fig. 2 clearly shows the overall proposed method and the attribute ranking based on the attributes explanation prior and the match between the saliency map and attribute activation maps.
- The attribute ranking and selection method of informative attributes using combinations weighted TCAV and cosine similarity between the attribute activation map and the generated saliency map is novel.
Weakness:
===========
- Applications of such a combined explanatory system don’t seem to be highly motivated in the introduction. I suggest the authors discuss more of the image similarity based applications and less on the discussion and heavy citation of generalized deep neural networks.
- The forms of the two loss components are both variants of l_{1} and l_{2} standard losses and they could be subject to issues with the standard variants of the l_{1} and l_{2} losses such as lack of translation and other transformation invariances. Hence, it would have been more useful to give the reasoning for the selection of the losses employed compared to other similarity and divergence based losses that are less sensitive to such variations.
- Similar to the above point, the choice of cosine similarity to compare match b/n attribute activation maps and saliency maps seem arbitrary. The method is described well but why cosine similarity was chosen in terms of its benefits compared to other similarity metrics is not that clear.
- Evaluation on more datasets such as person/pedestrian attributes datasets would have demonstrated the generalizability of the proposed method across multiple practical domains. As such, I would suggest the authors test their method on at least one person/pedestrian attributes dataset such as PETA, Market1501, etc.
- Although Fig. 1 (b) motivated a more practical high level explanation, in the results section, the attribute explanations are reduced to just the selected attribute that matched with the saliency well. Human-like concise attribute-based high level explanation just like the example given in Fig. 1 (b) would have made the paper stronger. Even if NLP is beyond the scope of this paper, a simple template based explanation that incorporated the selected/matched attribute would have been more effective.
- The results are too concise and a few ablation results on different losses etc. could have helped. There is too many qualitative results especially in the supplementary.
1) Bekele, E., Lawson, W. E., Horne, Z., & Khemlani, S. (2018). Implementing a Robust Explanatory Bias in a Person Re-identification Network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (pp. 2165-2172).


The long-tail distribution of the visual world poses great challenges for deep learning based classification models on how to handle the class imbalance problem. Existing solutions usually involve class-balancing strategies, e.g. by loss re-weighting, data re-sampling, or transfer learning from head-to tail-classes, but most of them adhere to the scheme of jointly learning representations and classifiers. In this work, we decouple the learning procedure into representation learning and classification, and systematically explore how different balancing strategies affect them for long-tailed recognition. The findings are surprising: (1) data imbalance might not be an issue in learning high-quality representations; (2) with representations learned with the simplest instance-balanced (natural) sampling, it is also possible to achieve strong long-tailed recognition ability at little cost by adjusting only the classifier. We conduct extensive experiments and set new state-of-the-art performance on common long-tailed benchmarks like ImageNet-LT, Places-LT and iNaturalist, showing that it is possible to outperform carefully designed losses, sampling strategies, even complex modules with memory, by using a straightforward approach that decouples representation and classification.

The paper tries to handle the class imbalance problem by decoupling the learning process into representation learning and classification, in contrast to the current methods that jointly learn both of them. They comprehensively study several sampling methods for representation learning and different strategies for classification. They find that instance-balanced sampling gives the best representation, and simply adjusting the classifier will equip the model with long-tailed recognition ability. They achieve start of art on long-tailed data (ImageNet-LT, Places-LT and iNaturalist).
In general, this is paper is an interesting paper. The author propose that instance-balanced sampling already learns the best and most generalizable representations, which is out of common expectation. They perform extensive experiment to illustrate their points.
--Writing:
This paper is well written in English and is well structured. And there are two typos. One is in the second row of page 3, "… a more continuous decrease [in in] class labels …" and the other one is in the first paragraph of section 5.4, "… report state-of-art results [on on] three common long-tailed benchmarks …".
--Introduction and review:
The authors do a comprehensive literature review, listing the main directions for solving the long-tailed recognition problem. They emphasis that these methods all jointly learn the representations and classifiers, which "make it unclear how the long-tailed recognition ability is achieved-is it from learning a better representation or by handling the data imbalance better via shifting classifier decision boundaries". This motivate them to decouple representation learning and classification.
--Experiment:
Since this paper decouples the representation learning and classification to "make it clear" whether the long-tailed recognition ability is achieved from better representation or more balanced classifier, I recommend that authors show us some visualization of the feature map besides number on performance. Because I am confused and difficult to image what "better representation" actually looks like.
The authors conduct experiment with ResNeXt-{10,50,101,151}, and mainly use ResNeXt-50 for analysis. Will other networks get similar results as that of ResNeXt-50 shown in Figure 1?
When showing the results, like Figure 1, 2 and Table 2, it would be better to mention the parameters chosen for \tau-normalization and other methods.
Conclusion:
I tend to accept this paper since it is interesting and renews our understanding of the long-tailed recognition ability of neural network and sampling strategies. What's more, he experiment is comprehensive and rigorous.


Meta-learning is an exciting and powerful paradigm that aims to improve the effectiveness of current learning systems. By formulating the learning process as an optimization problem, a model can learn how to learn while requiring significantly less data or experience than traditional approaches. Gradient-based meta-learning methods aims to do just that, however recent work have shown that the effectiveness of these approaches are primarily due to feature reuse and very little has to do with priming the system for rapid learning (learning to make effective weight updates on unseen data distributions). This work introduces Nodal Optimization for Recurrent Meta-Learning (NORML), a novel meta-learning framework where an LSTM-based meta-learner performs neuron-wise optimization on a learner for efficient task learning. Crucially, the number of meta-learner parameters needed in NORML, increases linearly relative to the number of learner parameters. Allowing NORML to potentially scale to learner networks with very large numbers of parameters. While NORML also benefits from feature reuse it is shown experimentally that the meta-learner LSTM learns to make effective weight updates using information from previous data-points and update steps.

This paper proposes a meta-learner that learns how to make parameter updates for a model on a new few-shot learning task. The proposed meta-learner is an LSTM that proposes at each time-step, a point-wise multiplier for the gradient of the hidden units and for the hidden units of the learner neural network, which are then used to compute a gradient update for the hidden-layer weights of the learner network. By not directly producing a learning rate for the gradient, the meta-learner’s parameters are only proportional to the square of the number of hidden units in the network rather than the square of the number of weights of the network. Experiments are performed on few-shot learning benchmarks. The first experiment is on Mini-ImageNet. The authors build upon the method of Sun et al, where they pre-train the network on the meta-training data and then do meta-training where the convolutional network weights are frozen and only the fully-connected layer is updated on few-shot learning tasks using their meta-learner LSTM. The other experiment is on Omniglot 20-way classification, where they consider a network with only full-connected layers and show that their meta-learner LSTM performs better than MAML.
The closest previous work to this paper is by Ravi & Larochelle, who also propose a meta-learner LSTM. The submission states about this work that “A challenge of this approach is that if you want to optimize tens of thousands of parameters, you would have a massive hidden state and input size, and will therefore require an enormous number of parameters for the meta-learner…In Andrychowicz at al. an alternative approach is introduced that avoids the aforementioned scaling problem by individually optimizing each parameter using a separate LSTM…” I don’t believe this is true.
As stated in the work by Ravi & Larochelle, “Because we want our meta-learner to produce updates for deep neural networks, which consist of tens of thousands of parameters, to prevent an explosion of meta-learner parameters we need to employ some sort of parameter sharing. Thus as in Andrychowicz et al. (2016), we share parameters across the coordinates of the learner gradient. This means each coordinate has its own hidden and cell state values but the LSTM parameters are the same across all coordinates.” Thus, Ravi & Larochelle employ something similar to Andrychowicz at al., meaning that the number of parameters in the LSTM meta-learner there is actually a constant relative to the size of the learner network.
Thus, the paper’s contribution relative to Ravi & Larochelle is to propose a LSTM meta-learner with more parameters relative to the learner model. Firstly, I think this comparison should be stated and explained clearly in the paper. Additionally, in order to prove the benefit of such an approach, I think a comparison to the work of Ravi & Larochelle with the exact experimental settings used in the submission (pre-training the convolutional network and only using the meta-learner LSTM to tune the last fully-connected layer) would be helpful in order to validate the usefulness of the extra set of parameters in their proposed meta-learner LSTM.
The submission also states that “In many cases it is preferred to have an inner loop that consists of multiple sequential updates. However the inner loop’s computational graph can become quite large if too many steps are taken. This often results in exploding and vanishing gradients since the outer loop still needs to differentiate through the entire inner loop (Aravind Rajeswaran (2019), Antoniou et al. (2018)). This limits MAML to domains where a small amount of update steps are sufficient for learning. The LSTM-based meta-learner proposed in this work, allow gradients to effectively flow through a large number of update steps. NORML can therefore be applied to a wide array of domains.” I think this statement should be validated if it is stated. Can it be shown that when making a lot of inner-loop updates, the LSTM meta-learner performs better than MAML because of overcoming the stated issues with differentiation through a long inner loop? The experiments done in the paper involve very few inner loop steps and so I don’t believe the claim is supported.
Lastly, the experimental results are not very convincing. Though the authors say they modify the method proposed in Sun et al, the results from Sun et al. are not shown in the paper. Sun et al actually seem to achieve better results than the submission with the same 1-shot and better 5-shot accuracy. Was there a reason these results are not shown in the submission? Additionally, for the Omniglot experiment, is there any reason why it was not performed with the typical convolutional network architecture? Since the original MAML results on 20-way Omniglot are with a convolutional network, using the convolutional network would make the results more meaningful relative to previous work and show that the method is more broadly applicable to convolutional networks.
I believe there are several issues with the paper as stated above. Because of these issues, it is hard to evaluate whether the idea proposed is of significant benefit.
References
Andrychowicz et al. Learning to learn by gradient descent by gradient descent. NIPS 2016.
Ravi & Larochelle. Optimization as a Model for Few-Shot Learning. ICLR 2017.
Sun et al. Meta-transfer learning for few-shot learning.


Given a set of distances amongst points, determining what metric representation is most "consistent" with the input distances or the metric that best captures the relevant geometric features of the data is a key step in many machine learning algorithms. In this paper, we focus on metric constrained problems, a class of optimization problems with metric constraints. In particular, we identify three types of metric constrained problems: metric nearness (Brickell et al. (2008)), weighted correlation clustering on general graphs (Bansal et al. (2004)), and metric learning (Bellet et al. (2013); Davis et al. (2007)). Because of the large number of constraints in these problems, however, these and other researchers have been forced to restrict either the kinds of metrics learned or the size of the problem that can be solved. We provide an algorithm, PROJECT AND FORGET, that uses Bregman projections with cutting planes, to solve metric constrained problems with many (possibly exponentially) inequality constraints. We also prove that our algorithm converges to the global optimal solution. Additionally, we show that the optimality error (L 2 distance of the current iterate to the optimal) asymptotically decays at an exponential rate. We show that using our method we can solve large problem instances of three types of metric constrained problems, out-performing all state of the art methods with respect to CPU times and problem sizes.

The paper presents an algorithm for optimizing an function f under the constraints that the square matrix variable x represents "metric". In this context, this means that we have also observed a graph G with n vertices, and x is of size n by n, x(i, j) < x(i, e) + x(e, j) if i ~ e and j ~ e are adjacent: this is a generalized for of triangle inequality.
Authors argue that the constraint "x is a metric" translate into exponentially many linear constraints, which results in to a hard to solve problem
The algorithm they propose to tackle this (Algorithm 1) has two subroutines that are shown in Algorithm 2 (Forget and Project). The Project subroutine itself is a projection onto a convex set according to a Bregman divergence, which is not trivial. In this paper I understand that authors only consider metrics of type x' L x where L = C'C >0 is psd

Authors claim that the sequence created by their algorithm asymptotically converges to the global optimum, and show numerical superiority to baselines.
Major remarks:
My general feeling is that the paper overstates its results. The paper has some good contribution, which could be better emphasized.
The algorithm stacks multiple subroutines which are not necessarily very light. I am skeptical about the numerical efficiency of such algorithms.
Theoretical results are stated asymptotically while interpreted in the text as finite steps results: page 5, after Corollary 1., read "The algorithm spends the first few iterations ..." in this case, a theoretical result should support the claim
The algorithm starts at a stationary point of f. This itself can be nontrivial. Can authors discuss this?
Minor remarks:
metric and distance to me mean the same, hence the first sentence of the intro doesn't read easily..
what is \cal A line 5 of Algorithm 1? It seems to be a "list of hyperplanes" according to the previous text, but it is unclear to me how to build it algorithmically
The notation L is confusing in Algo 1 MetricViolation: wasn't L the matrix defining the metric?
A few typos: l. 12 Algo 1, e = (i, j), 3.2 "global optimum [remove solution]."


Neural architecture search (NAS) searches architectures automatically for given tasks, e.g., image classification and language modeling. Improving the search efficiency and effectiveness have attracted increasing attention in recent years. However, few efforts have been devoted to understanding the generated architectures. In this paper, we first reveal that existing NAS algorithms (e.g., DARTS, ENAS) tend to favor architectures with wide and shallow cell structures. These favorable architectures consistently achieve fast convergence and are consequently selected by NAS algorithms. Our empirical and theoretical study further confirms that their fast convergence derives from their smooth loss landscape and accurate gradient information. Nonetheless, these architectures may not necessarily lead to better generalization performance compared with other candidate architectures in the same search space, and therefore further improvement is possible by revising existing NAS algorithms.

--- Update after author's response ---
Thank authors for providing the detailed response to all my concerns. I am revising my rating to weak accept.
Specifically, generalising with 52 different variants of depth-width settings are enough, and the updated version is more solid than the earlier one. I think this paper should be accepted.
Summary:
The paper observed one common pattern of searched cell by 5 NAS algorithms, which is the cell found usually has large width but small depth structure, and claims the reason of such pattern is because architectures with shallow but wide structure converge fast during training, and thus are sampled by the NAS policy. To justify this fast convergence claim, the paper proposed to 1) define a width-depth level (width based on feature maps dimension, and depth based on its DAG connection) for each cell in the search space, and randomly sampled one architecture at each level on top of some best-cell discovered by NAS algorithms, 2) training them on original task from scratch independently, and provide visualization of training curve under various learning rate settings, loss landscape as well as gradient variance plot. For theoretical analysis, the paper formulates the narrowest and widest cells, and showing the difference of gradient is bounded by its Lipschitz smoothness of parameter matrices, and usually such variance indicates the widest architecture could converge faster than the narrowest one.
Whilst this observation is interesting, I found the empirical and theoretical justifications seem to be insufficient to support the claims for the following reasons, 1) The experiments are overwhelmingly built on top of **1** architecture (even obtained from random sampling) of each width-depth level, and it may not well represent the common behavior in the search space, thus the generalization of these claims remains questionable; 2) Theoretical analysis showing the gradient variance difference of narrowest architecture is bounded comparing to the widest cell, however, in practice, these architectures are not properly evaluated. Without proper extension, it is hard to conclude that such difference bound between a wider and narrower architecture pair; 3) experiments in supplementary negatively affects the generalizability of this paper, since the observed trend on DARTS search space does not agree with the one on AmoebaNet and SNAS cases; 4) paper claims the NAS algorithm tends to pick the fast converging architectures more than those late converged one, while intuitive, without showing some detailed process about how NAS algorithms converge, and which architectures they actually sampled during the search phase.
Nevertheless, I do agree that this paper has a clear motivation, and the observation is interesting and important. If the author could show the conclusion still hold after scaling the experiments, I am learning to accept this paper in the end.
Strength
+ Observation of these NAS algorithms tends to pick wide but shallow cell type is interesting, motivation of this work is well justified.
+ Experiments are throughout, instructive under the paper's current setting.
+ Theoretical justification for the gradient variance between the narrowest and widest cells are sound.
+ Paper is well written and easy to follow, the figures are presented clearly.
Weakness
- Insufficient experimental setting to support the claim.
My major concern is that under the current experiment design, it is not clear if the observation is well justified, and impedes the main contribution. As mentioned in the review's summary, it is not that convincing that, for each width-depth level, one architecture is enough. I understand exploiting all the variants is resource consuming, however, without such experiments, the current experiments can be impacted by many factors, such as, 1) as in Appendix A2, for each level, the paper "fixed the partial order of their intermediate nodes" and "replace the source node of their associated operations by uniformly randomly sampling a node from their proceeding nodes in the same cell", if I understand correctly, this means the operations will remain the same. However, since these best architectures are searched over both operation and topology connection, new architectures generated from this way may be sub-optimal, hence the larger gradients or slow convergence is not only because they are "narrower" and "deeper". Without proper isolation, it is not possible to conclude as in paper.
To this end, I suggest author provide the following experiments, 1) sampling all the variants at each level, within a search space like NASBench-101[1], where all the architecture performances are known, 2) at least sampling a sufficient number in the current space (probably > 30 to be statistically significant); 3) Random sampling a small topological variances, then run NAS search algorithms to search the best operation set, then redo the experiments in the paper.
- Trending of DARTS evaluation results does not agree with SNAS and AmoebaNet.
In Figure  5 (loss landscape plot) and Figure 6 (gradient variance heatmap) for DARTS, the wider-shallower architectures are better comparing to narrower-deeper ones, however, this trend is not significant in Figure 14,16 for AmoebaNet space and Figure 15, 17 for SNAS space. After a closer look, I noticed in Figure 2, the Darts C3 and C4, input node x0 is not connected to the graph, while C, C1, C2, and all the topologies in Figure 10, 11, x0 is connected. Could the author(s) comment on this? Will this be the reason why the C3, C4 DARTS are worse than other architectures?
Minor comments
Page 1 - Introduction paragraph 3 line 1 - 'typologies': is this referring to 'topologies'?
Table 1 - Adapted architectures on CIFAR-10 are mostly worse, even on CIFAR-100, they are better in a small margin. This does not support the claim well.
Figure 9 - Could the author provide the width-depth information for each index?
Reference
[1] Ying et al. NAS-Bench-101: Towards Reproducible Neural Architecture Search, ICML'19.


Symbolic regression is a type of discrete optimization problem that involves searching expressions that fit given data points. In many cases, other mathematical constraints about the unknown expression not only provide more information beyond just values at some inputs, but also effectively constrain the search space. We identify the asymptotic constraints of leading polynomial powers as the function approaches 0 and ∞ as useful constraints and create a system to use them for symbolic regression. The first part of the system is a conditional production rule generating neural network which preferentially generates production rules to construct expressions with the desired leading powers, producing novel expressions outside the training domain. The second part, which we call Neural-Guided Monte Carlo Tree Search, uses the network during a search to find an expression that conforms to a set of data points and desired leading powers. Lastly, we provide an extensive experimental validation on thousands of target expressions showing the efficacy of our system compared to exiting methods for finding unknown functions outside of the training set.

This paper considers a task of symbolic regression (SR) when additional information called 'asymptotic constraints' on the target expression is given. For example, when the groundtruth expression is 3 x^2 + 5 x, it behaves like x^2 in the limit x -> infinity, thus the power '2' is given as additional information 'asymptotic constraint'. In the paper's setting, for SR with univariate groundtruth functions f(x), 'asymptotic constraints' for x-> infinity and x -> 0 are given. For this situation, the paper proposes a method called NG-MCTS with an RNN-based generator and MCTS guided by it to consider asymptotic constraints. In SR, a learner is asked to acquire an explicit symbolic expression \hat{f}(x) from a given set of datapoints {x_i, f(x_i)}, but unlike the parameter optimization aspect of a standard supervised ML setting, the problem is essentially combinatorial optimization over exponentially large space of symbolic expressions of a given context-free grammar (CFG). For a given symbolic space with a prespecified CFG, extensive experimental evaluations are performed and demonstrated significant performance gain over existing alternatives based on EA. Also, quantitative evaluations about extrapolative performance and detailed evaluation of the RNN generator are also reported.
Though it includes a lot of quite interesting methods and results, the paper has two major issues:
(1) the proposed method NG-MCTS explicitly uses the fact that the target expressions are generated from a CFG, but this assumption sounds unnatural if the target problem is SR (unlike the case of GVAE). Apparently, all results except a toy case study in 5.2 depend on artificial datasets from a CFG, and any practical advantages or impacts are still unclear because the experimental settings are very artificial.
(2) the most important claim of this paper would be the proposal to use 'asymptotic constraints', but the availability of this information sounds too strong and again artificial in practical situations. A one-line motivation saying that 'this property is known a priori for some physical systems before the detailed law is derived' is not convincing enough.


We study the problem of training machine learning models incrementally with batches of samples annotated with noisy oracles. We select each batch of samples that are important and also diverse via clustering and importance sampling. In particular, we incorporate model uncertainty into the sampling probability to compensate poor estimation of the importance scores when the training data is too small to build a meaningful model. Experiments on benchmark image classification datasets (MNIST, SVHN, and CIFAR10) shows improvement over existing active learning strategies. We introduce an extra denoising layer to deep networks to make active learning robust to label noises and show significant improvements.

Summary: The paper proposes an uncertainty-based method for batch-mode active learning with/without noisy oracles which uses importance sampling scores of clusters as the querying strategy. Authors evaluate their method on MNIST, CIFAR10, and SVHN against approaches such as Core-set, BALD, entropy, and random sampling and show superior performance.
Pros:
(+): The paper is well-written and well-motivated.
(+): The problem is timely and has direct real world applications.
(+): Applying the denoising layer is an interesting and viable idea to overcome noise effects.

Cons that significantly affected my score and resulted in rejecting the paper are as follows:

1 - Experimental setting and evaluations:
The biggest drawback in this paper is the experimental setting which is not rigorous enough for the following reasons:
(a) Weak datasets: Authors have chosen some standard benchmarks but they do not seem to be convincing as the datasets are too easy. Based on my experience, the behavior of an active learning agent trained on small number of classes does not necessarily generalize to cases where the number of classes is large. So I’d like to ask authors to try to evaluate on datasets with more number of classes as well as more realistic images (as opposed to thumbnail images).
(b) Comparison to state of the art: More importantly, authors are missing out on an important baseline which is a recent ICCV paper [1] on task-agnostic pool-based batch-mode active learning that has explored both noisy and perfect oracles and to the best of my knowledge is the current state of the art. Authors can extend their experimental setting to the datasets used in [1] including CIFAR100 and ImageNet and provide comparison. The reason that it is important to compare is that the method in [1] is task-agnostic and does not explicitly use uncertainty hence it is interesting to see how this method performs against it.
(c) More on baselines and related work: In addition to [1], different variation of ensemble methods have been serving as active baselines in this field and I recommend adding one as a baseline. For a recent work in this line you can see this paper from CVPR 2018 [2]. Moreover, the authors seem to be missing on a long-standing line of active learning research known as Query-by-Committee (QBC) began in 1997 [3] in the related work section which should be cited as well.
(d) Hyper parameter tuning: Last but not least about the experiments is the hyper parameter tuning which is not addressed. It is important to not use the well-known hyper parameters for these benchmarks that have been obtained using validation set from the entire dataset. Authors should explain how they have performed this.

2 - Report sampling time:
Another important factor missing in the evaluations is reporting time complexity or wall-clock time that it takes to query samples. Authors should measure this precisely and make sure it is being reported similarly across all the methods. I am asking this because random selection is still an effective baseline in the field and it only takes a few milliseconds. Therefore, the sampling time of a new algorithm should be gauged based on that while performing better than random. Given the multiple steps in this algorithm I am skeptical that the sampling time would be proportional to the gain obtained in accuracy versus labeling ratio over random selection baseline.

3: Section 5.2 is not informative:
(a) My last major concern is section 5.2 where the discussion on results is given along with supporting figures.
Lack of quantitative results: First of all, no quantitative results are given for the values plotted in figure 3 and 4 (neither in the main text nor in the supplement) and different methods happen to be too close to each other, making it hard to see the right color for standard deviations. Also, in the discussion corresponding to those figures no information is provided in this regard. It is important to report how much labeling effort this algorithm is saving by comparing number of samples needed by each method to achieve the same accuracy because that is the main goal in AL. Lack of numbers also makes it hard for this work to be used by others.
(b) Figure legends: The way authors have labeled their method in Figure 3 is confusing as the “Proposed+noise” happens to achieve better performance over “Proposed”. I think by “noise” authors meant denoising layer was being used (please correct me if I am wrong) but this is not what the legends imply.
(c) X axis label: It is common to report accuracy versus percentage of labeled data making it more understandable of how far each experiment has been through each dataset. Additionally, I recommend reporting the maximum achievable accuracy for each dataset assuming that all the data was labeled. This serves as an upper bound.
(d) Font sizes in figures: It will be helpful to make them larger.

4. I also have a more general concern about uncertainty-based methods. I know that they have been around for a long time but given the fact that predictive uncertainty is still an open problem and there is still no concrete method to measure calibrated confidence scores for outputs of a deep network (Dropout and BN given in this paper have been already outperformed by ensembles (see [4])), hence relying on uncertainty is not the best direction to go. It is literally chicken and egg problem to try to rely on confidence scores of the main-stream task while it is being trained itself. This issue has been raised in this paper but I am still not convinced that the paper has fully addressed it. I think the community needs to explore task-agnostic methods more deeply. [1] is a good start on this path but there is always more to do. This concern is not necessarily a major part of my decision assessment and I only want the authors to state their opinion on this and explain how accurately they think this issue is being addressed.

The following issues are less major and are given only to help, and not part of my decision assessment:
1- In Figure 3(c), it appears that the accuracy for “Proposed + noise” when \epsilon=0.1 is higher than when it is noise-free. It might be a miss-reading as the figure is coarse and it is hard to compare but if that is the case, can authors explain it?
2- The Abstract does not read well and does not state the main contribution. It has put too emphasize on batch-mode active learning which has become an intuitive approach since deep networks have become popular. Also the wording “Our approach bridges between uniform randomness and score based importance sampling of clusters” should be changed as all other active learning algorithms are trying to do that.
3 - In section 5.1 please state that you used VGG 16 (I assume so since it is what was used in the cited reference (Gal et al. 2017) but authors need to verify that. Also, the other citation given for this (Fchollet, 2015) is confusing as it is Keras package documentation while in the next sentence authors state that they have implemented their algorithm in PyTorch. So please shed some light on this.
*******************************************************************
As a final note, I would be willing to raise my score if authors make the experimental setting stronger (see suggestions above).
[1] Sinha, Samarth, Sayna Ebrahimi, and Trevor Darrell. "Variational Adversarial Active Learning." arXiv preprint arXiv:1904.00370 (2019).
[2] Beluch, William H., et al. "The power of ensembles for active learning in image classification." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018.
[3] Freund, Yoav, et al. "Selective sampling using the query by committee algorithm." Machine learning 28.2-3 (1997): 133-168.
[4] Lakshminarayanan, Balaji, Alexander Pritzel, and Charles Blundell. "Simple and scalable predictive uncertainty estimation using deep ensembles." Advances in Neural Information Processing Systems. 2017.
*******************************************************************
*******************************************************************
*******************************************************************
POST-REBUTTAL:
In the revised version, there are new tables (Table 1-4) provided in the appendix which I found too different than results reported for previous baselines by more than 6%. For example, according to Core-set paper (Sener, 2018), Figure 4, they achieve near 80% using 40% of the data (20000 samples), and according to VAAL paper (Sinha et al. 2019 github page: https://github.com/sinhasam/vaal/blob/master/plots/plots.ipynb), they achieve 80.90+-0.2. However, the current paper reports 71.99 ± 0.55 for Core-set, and 74.06 ± 0.47 for VAAL which is a large mismatch.
More importantly, looking at the results provided in VAAL paper (Sinha et al. 2019 or Core-set paper (Sener, 2018) they show their performance as well as most of their baselines is superior to random selection by a large gap, but in this paper results shown in Table 1 to 4 in almost all of them random is superior (or on-par) to all baselines and the proposed method is the only method that outperforms baseline which is clearly a wrong claim. Therefore, I decrease my score from weak reject to reject.


Reinforcement learning (RL) is frequently used to increase performance in text generation tasks, including machine translation (MT), notably through the use of Minimum Risk Training (MRT) and Generative Adversarial Networks (GAN). However, little is known about what and how these methods learn in the context of MT. We prove that one of the most common RL methods for MT does not optimize the expected reward, as well as show that other methods take an infeasibly long time to converge. In fact, our results suggest that RL practices in MT are likely to improve performance only where the pre-trained parameters are already close to yielding the correct translation. Our findings further suggest that observed gains may be due to effects unrelated to the training signal, concretely, changes in the shape of the distribution curve.

This paper first theoretically demonstrates that a commonly used reinforcement learning method for neural sequence-to-sequence models (e.g. in NMT), contrastive minimum risk training (CMRT), is not guaranteed to converge to local (let alone global) optima of the reward function. The paper then empirically demonstrates that the REINFORCE algorithm, while not subject to the same theoretical flaws as CMRT, in practice fails to improve NMT models unless the baseline model is already "nearly correct" (i.e. the correct tokens were already within the few most probable tokens before the fine-tuning steps with REINFORCE). In fact, some of the performance gains of using REINFORCE/CMRT can be attributed to making the model's output probability distribution more peaked, and not necessarily from making the target tokens more probable as commonly assumed.
Overall, this is an excellent paper that offers significant contributions for the field. I have summarised the key strengths of the paper below, along with several suggestions and questions that I hope will be addressed by the authors. Based on my assessment, I am recommending a rating of "Accept" for this paper.
Strengths:
1. The paper is very well-written and well-structured. It starts off by pointing out the theoretical limitations of CMRT (and concisely recaps the key differences between CMRT and REINFORCE), and continues with an extensive set of experiments that clearly illustrates the limitations of REINFORCE in practice.
2. I also like the use of both controlled simulations (including one where the reward is constant) and NMT experiments with real data. The controlled simulations are useful to abstract away from the full complexity of the model and investigate what happens under various control scenarios, while the NMT experiments demonstrate that the findings still hold under the realistic setup.
3. The findings are really interesting and clearly illustrate the limitations of existing REINFORCE/CMRT methods for neural sequence-to-sequence models. It is very interesting to see that REINFORCE fails to make the target token most probable when the initial model ranks the target token as the third or more probable tokens under the model (Figure 2), even under the simple controlled simulations, which highlights the prohibitively high sample complexity of the model.
4. The peakiness effect hypothesis (i.e. attributing the gains of REINFORCE to making the output distribution more peaked, and not necessarily by making the target tokens more probable) is well-supported by the paper's empirical evidence. It is really illuminating that using a constant reward of 1 leads to the same BLEU score as actually optimising for BLEU in NMT (Section 5.2).
Suggestions and questions:
1. Section 4.2 (NMT Experiments) indicates that REINFORCE fine-tuning is done for 10 epochs, with 5,000 sentences per epoch, and k=1. Considering the enormous discrete sample space, one could expect that using multi-sample REINFORCE (i.e. k > 1) and training the model for many more epochs might mitigate the identified problems to some extent, and thus change the findings. Training for 5,000 sentences * 10 epochs may just not be enough for the RL fine-tuning to make a big difference.
2. In Figure 1, the x-axis is the "Update Size" with a scale between -1.0 and 1.0. This "Update Size" variable is not really explained in the paper, and why the scale is between -1.0 and 1.0.
3. In my understanding, the controlled simulations (Section 4.1) is done at the word-level (including word-level rewards, as opposed to the NMT experiments which are done at the sequence-level with sentence-level rewards). If this is the case, this should be made clearer.
4. To make Figure 3 easier to understand, the caption should indicate that a lower cumulative percentage means a more peaked output distribution.
5. Rather than breaking down the analysis by where the target token is ranked by the initial, pre-RL model (e.g. the target token is ranked second/third best in Figures 2 and 5), perhaps what really matters is the probability assigned to the target token. For instance, even if the target token is ranked third best by the initial model, there will be a big difference whether it is assigned a probability of 0.1 or 0.01 (i.e. the latter case is much less likely to be sampled, which would exacerbate the problem). Including this analysis might help strengthen the paper further.


An open question in the Deep Learning community is why neural networks trained with Gradient Descent generalize well on real datasets even though they are capable of fitting random data. We propose an approach to answering this question based on a hypothesis about the dynamics of gradient descent that we call Coherent Gradients: Gradients from similar examples are similar and so the overall gradient is stronger in certain directions where these reinforce each other. Thus changes to the network parameters during training are biased towards those that (locally) simultaneously benefit many examples when such similarity exists. We support this hypothesis with heuristic arguments and perturbative experiments and outline how this can explain several common empirical observations about Deep Learning. Furthermore, our analysis is not just descriptive, but prescriptive. It suggests a natural modification to gradient descent that can greatly reduce overfitting.

The paper studies the link between alignment of the gradients computed on different examples, and generalization of deep neural networks. The paper tackles an important research question, is very clearly written, and proposes an insightful metric. In particular, through the lenses of the metric it is possible to understand better the learning dynamics on random labels. However, the submission seems to have limited novelty, based on which I am leaning towards rejecting the paper.
Detailed comments
1. The prior and concurrent work is not discussed sufficiently:
a) The novelty of the "Coherent Gradients hypothesis" is not clear to me. First, the empirical fact that some examples are easier to learn than others in training of deep networks was the key focus of [5].
Hence, "Coherent Graident Hypothesis" should be mostly considered an explanation for why simple examples are/simple function are learned first. "Coherent Gradient Hypothesis" proposes that the key mechanism behind this phenomena is that simple examples/functions have co-aligned gradients and hence a larger "effective" learning rate. However, there are already quite convincing and closely related hypotheses. For example, the spectral bias interpretation of deep networks [2] and (2) suggests the same view actually. Just expressed in a different formalism, but can be also casted as having a higher effective learning rate for the strongest modes. Similarly, [3] proposes that SGD learns functions of increased complexity. A detailed comparison between these hypotheses is needed.
b) "Gradient coherence" metric is very closely related to Stiffness studied in [1] (01.2019 on arXiv). [1] studies the cosine (or sign) between gradients coming from different examples, and reach quite similar conclusions. It is also worth noting that [6, 7] propose and study a very similar metric as well. While arXiv submissions is not consider prior work, these three preprints should be discussed in detail in the submission.
c) It should be also remarked that "Coherent Gradient hypothesis" is to some extend folk knowledge. In particular, it is quite well known and also brought to the attention of the deep learning community that in linear regression strongest modes of the datasets as learned first when training using GD (see for instance [4]), which causally speaking stems directly from gradient coherence; these modes correspond to the largest eigenvalues of the (constant) Hessian. To make it more precise: consider that GD solving linear regression can be seen as having higher "effective" learning rates along the strongest modes in the dataset.
2. Experiments on random labels and restricting gradient norms are interesting. However, [5] should be cited. They experimented with regularization impact on memorization, which due to the addition of noise, probably also supresses weak gradients.
3. Experiments on MNIST do not feel adequate. While I do not doubt the validity of the experimental results, the paper should include results on another dataset; ideally from other domain than vision.
4. Plots in Figure 4 are too small to read. I would recommend moving half of them to the Supplement?
5. "Understanding why solutions of the optimization problem on the training sample carry over to the population at large" - Not sure what do you mean here. Could you please clarify?
6. "Furthermore, while SGD is critical for computational speed, from our experiments and others (Keskar et al., 2016; Wu et al., 2017; Zhang et al., 2017) it appears not to be necessary.". Please note there is very little work on training with GD large models. Also, citing in this context Keskar is misleading. Wasn't the whole point of Keskar to show why large batch size training overfits? Finally, there are many papers on studying the role of learning rate and batch size in generalization (not computational speed). I think this sentence should be rewritten to clarify what is the experimental data that GD is "sufficient", and SGD is just needed for "computational speed".
References
[1] Stanislav Fort et al, Stiffness: A New Perspective on Generalization in Neural Networks, https://arxiv.org/abs/1901.09491
[2] Rahaman et al, On the Spectral Bias of Neural Networks, https://arxiv.org/abs/1806.08734
[3] Nakkiran et al, SGD on Neural Networks Learns Functions of Increasing Complexity, https://arxiv.org/abs/1905.11604
[4] Goh, Why Momentum Really Works, https://distill.pub/2017/momentum/
[5] Arpit et al, A Closer Look at Memorization in Deep Networks, https://arxiv.org/abs/1706.05394
[6] He and Su, The Local Elasticity of Neural Networks, https://arxiv.org/abs/1910.06943
[7] Sankararaman, The Impact of Neural Network Overparameterization on Gradient Confusion and Stochastic Gradient Descent, https://arxiv.org/abs/1904.06963


A weakly supervised learning based clustering framework is proposed in this paper. As the core of this framework, we introduce a novel multiple instance learning task based on a bag level label called unique class count (ucc), which is the number of unique classes among all instances inside the bag. In this task, no annotations on individual instances inside the bag are needed during training of the models. We mathematically prove that with a perfect ucc classifier, perfect clustering of individual instances inside the bags is possible even when no annotations on individual instances are given during training. We have constructed a neural network based ucc classifier and experimentally shown that the clustering performance of our framework with our weakly supervised ucc classifier is comparable to that of fully supervised learning models where labels for all instances are known. Furthermore, we have tested the applicability of our framework to a real world task of semantic segmentation of breast cancer metastases in histological lymph node sections and shown that the performance of our weakly supervised framework is comparable to the performance of a fully supervised Unet model.

This paper proposes a new type of weakly supervised clustering / multiple instance learning (MIL) problem in which bags of instances (data points) are labeled with a "unique class count (UCC)*, rather than any bag-level or instance-level labels.  For example, a histopathology slide (the bag), consisting of many individual pixels to be labeled (the instances) could be labeled at the bag level only with UCC = 1 (for only healthy or only metastatic) or UCC = 2 (for mixed / border case).  The paper then proposes an approach for clustering instances based on the following two-step approach: (1) a UCC model is trained to predict the UCC given an input bag, and (2) the features of this learned UCC model are used in an unsupervised clustering algorithm to the get the instance-level clusters / labels.  The paper also provides a theoretical argument for why this approach is feasible.
Overall this paper proposes a (1) novel and creative approach that is well validated both (2) theoretically and (3) empirically, and relevant to a real-world problem, and thus I believe should be accepted.  In slightly more detail:
- (1) MIL where we are given bag-level labels only is a well-studied problem that occurs in many real world settings, such as the histopathology one used in this paper.  However to this reader's knowledge, this is a new variant that is both creative and motivated by an actual real-world study, which is exciting and alone warrants presentation at the conference in my opinion.
- (2) The theoretical treatment is high-level, but still serves a clear purpose of establishing feasibility of the proposed method- this modest and appropriate purpose serves the paper well.
- (3) The empirical results are thorough---e.g. the use of the two loss components for the UCC model are appropriately ablated, there are a range of baseline approaches compared to, multiple evaluation points are provided (i.e. both UCC prediction and final clustering metrics), and a real world use case is presented--and the results are impressive.
Some comments to improve the paper:
- The connection of the proposed UCC approach to the motivating histopath example should be explicitly stated upfront to help the reader understand how this method could be used and how it makes sense!
- It seems that the KDE element of the UCC model was chosen in part to enable the theoretical analysis?  If so, this should be clearly stated to help the reader understand the design rationale.
- Either way, it seems that a different approach than the KDE layer could have been taken- this should be added to the ablation experiments


We present a novel approach G-PATE for training a scalable differentially private data generator, which can be used to produce synthetic datasets with strong privacy guarantee while preserving high data utility. Our approach leverages generative adversarial nets to generate data and exploits the PATE (Private Aggregation of Teacher Ensembles) framework to protect data privacy. Compared to existing methods, our approach significantly improves the use of privacy budget. This is possible since we only need to ensure differential privacy for the generator, which is the part of the model that actually needs to be published for private data generation. In particular, we connect a student generator with an ensemble of teacher discriminators and propose a private gradient aggregation mechanism to ensure differential privacy on all the information that flows from the teacher discriminators to the student generator. Theoretically, we prove that our algorithm ensures differential privacy for the generator. Empirically, we provide thorough experiments to demonstrate the superiority of our method over prior work on both image and non-image datasets.

This paper studies the problem of differentially private data generator. Inspired by the general GAN framework and the PATE mechanism, the authors propose a new differentially private training algorithm for data generator. The problem of training data generator with privacy guarantee considered in this paper is very interesting, and the proposed algorithm looks novel. However, there are lots of unclear statements in the current paper, and I cannot tell whether the proposed algorithm is indeed better than previous methods. Following are my major concerns:
1.It is unclear what are the loss functions used in equation (1) and (2). Please define k when introducing equation (2).
2.The training framework introduced in section 3.1 is different from the traditional GAN framework, and thus my concern is that whether this framework will give us good generated samples. Because the performance of GAN has been proved in both theory and practice. The authors should at least empirically show the performance of the proposed framework in the nonprivate setting.
3.There is no introduction of the $(\epsilon,\delta)$-differential privacy before introducing the Definition 1.
4.There is no definition of Renyi differential privacy, so the statement of Theorem 2 is unclear. In addition, what is data-dependent Renyi differential privacy?
5.The privacy guarantee of Algorithm 2 is not very clear. Because there are lots of parameters in Algorithm 1 and 2 which may affect the privacy guarantee, and Theorem 3 does not state such requirements. For example, how to choose $\sigma_1,\sigma_2$? In Theorem 7, there are some constraints on different parameters, will them be satisfied by your algorithm?
6.How will the number of teacher models affect the privacy guarantee?
7.Why you choose random projection matrix with variance $1/k$, and what is the projection dimension for different algorithms?
8.In Table 1, the results of non private GAN are different from the results of non private GAN reported in PATE-GAN paper. Since the baseline results are much better in the current than the results reported in the PATE-GAN paper, it seems to me that the improvements of the proposed method comes from the stronger baseline.
Other comments:
1.$\lambda>1$ in Theorem 3.
2.Algorithm 2 should be moved to main context.
3.The last sentence in section 3.2 is not convincing.
4.Typo “differnet” in the caption of Table 1.


We present a selective sampling method designed to accelerate the training of deep neural networks. To this end, we introduce a novel measurement, the minimal margin score (MMS), which measures the minimal amount of displacement an input should take until its predicted classification is switched. For multi-class linear classification, the MMS measure is a natural generalization of the marginbased selection criterion, which was thoroughly studied in the binary classification setting. In addition, the MMS measure provides an interesting insight into the progress of the training process and can be useful for designing and monitoring new training regimes. Empirically we demonstrate a substantial acceleration when training commonly used deep neural network architectures for popular image classification tasks. The efficiency of our method is compared against the standard training procedures, and against commonly used selective sampling alternatives: Hard negative mining selection, and Entropy-based selection. Finally, we demonstrate an additional speedup when we adopt a more aggressive learningdrop regime while using the MMS selective sampling method.

### Summary of contributions
This paper aims to accelerate the training of deep networks using a selective sampling.
They adapt ideas from active learning (which use some form of uncertainty estimation about the class of the label) to selectively choose samples on which to perform the backward pass. Specifically, they use the minimal margin score (MMS).
Their algorithm works by computing the forward pass over a batch of size B (which is much larger than the regular batch of size b), compute the uncertainty measure for each sample, and only perform the backward pass over the b samples with the highest uncertainty. The motivation is that the backward pass is more expensive than the forward pass, and that by only performing this pass on a subset of samples, computations are saved.
### Recommendation
Reject. The central premise of the paper is unclear, the writing/presentation needs improvement, and the experiments are not convincing.
### Detailed comments/improvements:
There is a central premise of the paper that I don't understand: that the forward pass is much cheaper than the backward pass.
This is claimed in the intro by referring to charts that hardware manufacturers publish (but there are no references included), but I don't see why this should be the case.
For a linear network with weights W, the forward pass is given by the matrix-matrix product (rows of X are minibatch samples):
Y = XW^T
and the backward pass is given by the two matrix-matrix products:
dL/dX = dL/dY*dY/dX = dL/dY*W
dL/dW = dL/dY*dY/dW = dL/dY*X^T
Similarly the two operations in the backward pass for convolutional layers are given by a convolution of the output gradients with the transposed weigtht kernels and the input image respectively.
Point being, I don't see why the backward pass should be more than 3x more expensive than the forward pass. A simple experiment in PyTorch confirms this: the code snippet pasted at the bottom shows that the backward pass takes only around 2.6x longer than the forward pass.
fprop: 0.009286s
bprop: 0.0240s
bprop/fprop: 2.5893x
In algorithm 1, it is assumed that b << B. For this to be effective the forward pass would have to be *much* faster than the backward pass for this method to yield an improvement in computation. Can the authors comment on where this justification comes from?
I am unclear on what the purpose of Section 4.1 is. This shows that the MMS of the proposed method is lower than the other two, but this should be completely expected since that is exactly the quantity being minimized.
There are also several unsubstantiated claims: "Lower MMS scores resemble a better...batch of samples", "the batches selected by our method provide a higher value for the training procedure vs. the HNM samples.", "Evidently, the mean MMS provides a clearer perspective...and usefulness of the selected samples". What does higher value, usefulness, clearer perspective mean?
More generally, it is unclear if there is really any improvement in the final performance from using the proposed method.
In Figure 2, all methods seem to have similar final performance.
In Figure 5, is there a reason why the curve for MMS is cut off? How does its final performance compare to that of the baseline method in red? It looks like the baseline might be better, but it's hard to tell from the figure.
Why are the experiments with the entropy measure in a seperate section? Please include them along with the other methods in the same plot, i.e merge Figure 2 and Figure 4.
My suggestions for improving the experimental section are as follows:
- include all methods together in all the plots/tables
- repeat experiments multiple times with different seeds to get error bars. Include these both in the learning curves and in the tables.
- It's hard to see small differences in the learning curves, so including tables as well is important. Include best performance for all the methods in the tables.
Finally, in 2019 CIFAR alone is not longer a sufficient dataset to report experiments on. Please report results on ImageNet as well.
One of the central premises of the paper is acceleration in terms of compute/time. To make this point, there should also be results in terms of walltime and floating-point operations. Please include these results in the paper.

### Code snippet timing forward/backward passes
import torch, torch.nn as nn, time
model =	nn.Sequential(nn.Linear(784, 1000),
                      nn.ReLU(),
                      nn.Linear(1000, 1000),
                      nn.ReLU(),
                      nn.Linear(1000, 10),
                      nn.LogSoftmax())
data = torch.randn(128, 784)
labels = torch.ones(128).long()
t = time.time()
pred = model.forward(data)
loss = nn.functional.nll_loss(pred, labels)
fprop_time = time.time() - t
t = time.time()
loss.backward()
bprop_time = time.time() - t
print('fprop: {:.4}s'.format(fprop_time))
print('bprop: {:.4f}s'.format(bprop_time))
print('bprop/fprop: {:.4f}x'.format(bprop_time / fprop_time))


Designing a convolution for a spherical neural network requires a delicate tradeoff between efficiency and rotation equivariance. DeepSphere, a method based on a graph representation of the sampled sphere, strikes a controllable balance between these two desiderata. This contribution is twofold. First, we study both theoretically and empirically how equivariance is affected by the underlying graph with respect to the number of vertices and neighbors. Second, we evaluate DeepSphere on relevant problems. Experiments show state-of-the-art performance and demonstrates the efficiency and flexibility of this formulation. Perhaps surprisingly, comparison with previous work suggests that anisotropic filters might be an unnecessary price to pay.

The paper presents DeepSphere, a method for learning over spherical data via a graphical representation and graph-convolutions. The primary goal is to develop a method that encodes equivariance to rotations, cheaply. The graph is formed by sampling the surface of the sphere and connecting neighbors according to a distance-based similarity measure. The equivariance of the representation is demonstrated empirically and theoretical background on its convergence properties are shown. DeepSphere is then demonstrated on several problems as well as shown how it applies to non-uniform data.
The paper is interesting and clear. The projection of structured data to graphical representations is both efficient in utilizing existing algorithmic techniques for graph convolutions and useful for approaching the spherical structure of the data. The theoretical analysis and discussion of sampling is interesting, though should be more clearly stated throughout and potentially visualized in figures.
The experiments performed are thorough and interesting. The approach both outperforms baselines in inference time and accuracy. However, one wonders the performance on the well-researched tasks such as the performance on 3D imagery, e.g., Su & Grauman, 2017; Coors et al., 2018.
The unevenly sampled data is a nice extension showing the generality of the approach. How does the approach work for data connected within a radius rather than a k-nearest approach?
Minor:
- A figure detailing the parameters and setup for theorem 3.1 and figure 2 would be useful.
- The statement on the dispersion of the sampling sequence states “the smallest ball in \R^3 containing \sigma_i”, but I believe it should be “containing only \sigma_i”.


Deep learning's success has led to larger and larger models to handle more and more complex tasks; trained models can contain millions of parameters. These large models are compute-and memory-intensive, which makes it a challenge to deploy them with minimized latency, throughput, and storage requirements. Some model compression methods have been successfully applied on image classification and detection or language models, but there has been very little work compressing generative adversarial networks (GANs) performing complex tasks. In this paper, we show that a standard model compression technique, weight pruning, cannot be applied to GANs using existing methods. We then develop a selfsupervised compression technique which uses the trained discriminator to supervise the training of a compressed generator. We show that this framework has a compelling performance to high degrees of sparsity, can be easily applied to new tasks and models, and enables meaningful comparisons between different pruning granularities.

In this paper, the authors tackle the task of compressing a network. While there
are many effective solutions so far regular computer vision tasks, as they demonstrate,
they fail catastrophically  when applied to generative adversarial networks(GANs).
They propose a modification to the classic distillation method, where a
"student" network tries to imitate the uncompressed one under the supervision of
a fully converged discriminator network. They perform evaluation on multiple
tasks from image synthesis to super-resolution. They also study the influence
of the compression factor on the quality of the generated images.
The task is well motivated and situated in the related literature. The first
section is very thorough and extremely efficient at describing the failure modes
of existing methods. On one side, the results demonstrated in the evaluation are
compelling, on the other side, the compression factor is only 50%, which is much
lower than seen in related work. However, as it is shown in section 3 the task
may be much harder for GANS than regular models so I still consider it a
sizeable contribution.
There are a couple of points that require clarification. I personally found the
description of the method (Section 4) rather confusing. It is clear
what "discriminative loss" is as it is the one used in every GAN.
Unfortunately, I could not understand what "generative loss" means in the general
case. An example is given for StarGAN in equation (7) and I have a rough idea of
what to choose for Style Transfer, Domain Translation, Super Resolution and
Image translation. Though, it is unclear to me what to use in the case of
image synthesis. The experiments clearly show that it is possible so I think
it is necessary to show how this framework is concretely applied to each task at
hand.
During training, the discriminator only ever saw pictures from the true distribution
and the distribution generated by the generator (at each of its training steps).
If I understood the framework properly, here, the compressed generator is trained
from a random initialization. The distribution it outputs is therefore
completely unknown and potentially non overlapping with either of the true or
the generator ones. In that case it is hard to predict what the discriminator
would do on completely out of distribution samples. I seems reasonable to
conjecture that it might consider them "true" because it was never trained on
them. Could you provide an explanation of why it is not a problem in practice?
Do you have to try multiple initializations? Is the generative
loss enough to force the compressed discriminator to match the support
of the distribution of the dense generator?
I think this paper is novel, tackles a hard task and presents compelling results
(albeit using very mild compression ratios). It should be accepted if some
clarifications are made in section 3.
Minor Remarks:
- Figures 2, 9, 42 and 43 are unreadable when printed with a regular color office
  printer.
- It is unclear what it would take an extra 10% of the original number of epochs to train the compressed network. Why couldn't it be faster, or much longer?


Generalized zero-shot learning (GZSL) is the task of predicting a test image from seen or unseen classes using pre-defined class-attributes and images from the seen classes. Typical ZSL models assign the class corresponding to the most relevant attribute as the predicted label of the test image based on the learned relation between the attribute and the image. However, this relation-based approach presents a difficulty: many of the test images are predicted as biased to the seen domain, i.e., the domain bias problem. Recently, many methods have addressed this difficulty using a synthesis-based approach that, however, requires generation of large amounts of high-quality unseen images after training and the additional training of classifier given them. Therefore, for this study, we aim at alleviating this difficulty in the manner of the relation-based approach. First, we consider the requirements for good performance in a ZSL setting and introduce a new model based on a variational autoencoder that learns to embed attributes and images into the shared representation space which satisfies those requirements. Next, we assume that the domain bias problem in GZSL derives from a situation in which embedding of the unseen domain overlaps that of the seen one. We introduce a discriminator that distinguishes domains in a shared space and learns jointly with the above embedding model to prevent this situation. After training, we can obtain prior knowledge from the discriminator of which domain is more likely to be embedded anywhere in the shared space. We propose combination of this knowledge and the relationbased classification on the embedded shared space as a mixture model to compensate class prediction. Experimentally obtained results confirm that the proposed method significantly improves the domain bias problem in relation-based settings and achieves almost equal accuracy to that of high-cost synthesis-based methods.

Summary:
This paper proposes a relation-based ZSL model which can effectively alleviate the domain bias problem. To this end, first, the paper claims that a good relation-based ZSL model should consider two requirements -- modality invariance and class separability. And the paper designed Modality-invariant and Class-separable Multimodal VAE (MCMVAE) based on VAEs to meet the two aforementioned requirements. Next, the paper hypothesizes that the domain bias problem is due to the overlap between seen and unseen classes in the shared space, and explicitly introduced a discriminator to separate the two domains. The paper performs experiments on ZSL benchmark datasets and shows that the proposed method outperforms other relation-based methods. Besides, the domain discriminator which can be applied to other models demonstrates its effectiveness in reducing domain bias given the experimental results.
+Strengths:
1. Clear writing logic. The author clearly depicts how to get the final loss of the method step-to-step and the relationship with existing methods.
2. The version without the domain discriminator (i.e. MCMVAE) is similar to PSE and CADA-VAE as the author acknowledges. However, the domain discriminator has certain novelty and can be applied to other methods. The overlap among seen and unseen classes is an important problem (domain bias problem named by the author) and the add of the domain discriminator to distinguish whether a sample is from seen classes or unseen classes is reasonable, which can provide better class separability (among seen and unseen classes).
-Weaknesses:
1. Although the author claims that the proposed method is a relation-based method, it is strange that the proposed method is called xxVAE but in Table 2 it doesn't fall into synthesis-based methods (as CVAE-ZSL and CADA-VAE do). Although it is derived from VAE, the current method doesn't seem to be called a VAE any more (some of the regularizations of the VAE are relaxed). Also, are the two terms -- relation-based and synthesis-based -- first proposed by the author? Is there a clear boundary between those two groups of methods?
2. It is recommended that an additional figure that depicts the framework is added (similar to Figure 2 in CADA-VAE) to promote better understanding. Currently, the method part only contains formulas with many parameters, making it difficult to grasp the idea of the whole framework at first glance.
3. The novelty of this paper is somewhat limited while missing some relevant works, e.g.[r1, r2]. [r1] learns a latent space where the compactness within class and separateness between classes are considered. [r2] uses a two-stage prediction for GZSL.
[r1] Jiang et al. Learning Discriminative Latent Attributes for Zero-Shot Classification. In IEEE ICCV 2017.
[r2] Zhang et al. Model Selection for Generalized Zero-shot Learning. In arXiv 2018.
4. It is a question whether the seen and unseen classes can be separated (Whether a two stage process is correct?). The key for ZSL is knowledge transfer and the base is that seen and unseen classes are related [r3]. If they are separated, can one use the model trained on seen classes to recognize the unseen classes? This is quite problematic. Besides, in Tab.2 there lacks of necessary comparisons with recent relation-based approaches e.g.[r3][r4], which makes the evaluation less sufficient.
[r3] Jiang et al. Transferable Contrastive Network for Generalized Zero-Shot Learning. In IEEE ICCV 2019.
[r4] Li et al. Discriminative Learning of Latent Features For Zero-Shot Recognition. In IEEE CVPR 2018.
5. Some unclear/incorrect descriptions of the method:
5.1) The formulation of GZSL is incorrect. Y= union(y_s  y_u), but not intersection(y_s  y_u)
5.2) How is the class separation formulated in the framework?
5.3) In Sec.3.2, why is the log-likelihood of the generative models can be obtained by the L1 loss?
Minor issues:
1. Better use vectorgraphs for clear view (especially for Figure 3 and 4).
2. Incomplete reference: for Probabilistic semantic embedding (PSE), the reference should add the conference information.
3. Grammar and spelling mistakes:
[1] Content in Figure 2 (not caption): unseen class -> unseen classes
[2] Last line in 4.1: MCVAE-D -> MCMVAE-D
[3] Last paragraph in 4.2: close -> stay close
[4] Last model name in Table 1: MCMVAE -> MCMVAE-D
4. The color bar for the contours at the rightmost of Figure 3 is not clear (not the standard way to draw a color bar, better refer to what a color bar is usually drawn).
5. If possible, better reduce the main text to 8 pages as recommended by the submission instructions (e.g. some content of the method part can be moved to the appendix?).