Activation in deep neural networks is fundamental to achieving non-linear mappings. Traditional studies mainly focus on finding fixed activations for a particular set of learning tasks or model architectures. The research on flexible activation is quite limited in both designing philosophy and application scenarios. In this study, we propose a general combined form of flexible activation functions as well as three principles of choosing flexible activation component. Based on this, we develop two novel flexible activation functions that can be implemented in LSTM cells and auto-encoder layers. Also two new regularisation terms based on assumptions as prior knowledge are proposed. We find that LSTM and fully connected auto-encoder models with proposed flexible activations provides significant improvements on time series forecasting and image compressing tasks, while layer-wise regularization can improve the performance of CNN (LeNet-5) models with PReLu activation in image classification tasks.

This paper presents a family of parameterized composite activation functions, and a regularization technique for parameterized activation functions in general. While the three sets of experiments show potentially promising results, they aren't able to disambiguate clearly between the effect of the activation function you introduce and the effect of additional parameters in general, or between the effect of the regularization technique you introduce and the effect of regularization in general. I would really like to see the kind of carefully baselined, ablated, and hyperparameter-tuned results that would justify adding the techniques introduced to the toolbox of a typical deep learning practitioner.
Some feedback:
Typos:
- The formatting is off a bit (shifted horizontally?)
- In the abstract: RPeLu -> PReLU
- p. 5: "basement settings"->baseline, "logarithmic sale"->scale
- p. 6: basement->baseline again
- p. 7 etc.: trail->trial
Feedback:
- Intro:
  - Explain why maxout is similar to training piecewise activation functions?
  - It's unclear what "making the most of the non-linear properties by introducing adaptation policy on the input" means
  - An "autoencoder" is not an architecture as much as a broad family of architectures coupled with training approaches (I'm guessing you mean a fully-connected autoencoder)
- Methodology:
  - consider using "f" instead of "a" so it's easier to tell apart from alpha?
- Experiments:
  - I'm not sure I'm convinced by the statistical tests used on the LSTM results; they demonstrate that your approach, with a few specific sets of hyperparameter settings, does better than the baseline, but not that this represents a valid claim about your activation function's effect on this LSTM model in general.
  - The autoencoder experiments are even less convincing, in that they represent two seemingly arbitrary network architectures, with equally arbitrary placement of the activation function in one of them.
  - The regularization experiments use LeNet-5, which is not a compelling benchmark architecture with respect to contemporary practice. The effects of regularization techniques can be very different in different regimes of dataset and network size.
- Code:
  - I'm not sure why you performed backprop by hand for your activation functions and used torch.Function, rather than writing them directly as nn.Modules that make use of PyTorch autograd
  - There are lots of details in the code that aren't in the paper; in general, papers should aim to be relatively self-contained (although I'm very glad to see your code, and it's pretty simple to follow)


The need for labour intensive pixel-wise annotation is a major limitation of many fully supervised learning methods for image segmentation. In this paper, we propose a deep convolutional neural network for multi-class segmentation that circumvents this problem by being trainable on coarse data labels combined with only a very small number of images with pixel-wise annotations. We call this new labelling strategy 'lazy' labels. Image segmentation is then stratified into three connected tasks: rough detection of class instances, separation of wrongly connected objects without a clear boundary, and pixel-wise segmentation to find the accurate boundaries of each object. These problems are integrated into a multitask learning framework and the model is trained end-to-end in a semi-supervised fashion. The method is demonstrated on two segmentation datasets, including food microscopy images and histology images of tissues respectively. We show that the model gives accurate segmentation results even if exact boundary labels are missing for a majority of the annotated data. This allows more flexibility and efficiency for training deep neural networks that are data hungry in a practical setting where manual annotation is expensive, by collecting more lazy (rough) annotations than precisely segmented images.

The submission presents a neural network for multi-task learning on sets of labeled data that are largely weakly supervised (in this case, partially segmented instances), augmented by comparatively fewer fully supervised annotations.
The multiple tasks are designed to make use of both the weak as well as as full (‘strong’) labels, such that performance on fully annotated machine-generated output is improved.
As noted in the related work section (Section 2), multi-task methods aim to use benefits from underlying common information that may be ignored in a single-task setting. The network presented here is quite similar to most of these multi-task approaches: a common feature encoder, and partially distinct feature decoding and classification parts.
The (minor) novelty mainly comes from the distinct types of weak/strong annotation data fed here: instance scribbles, boundary scribbles, and (some or few) full segmentations.
The submission is overall well written and provides sufficient clarity and a good overview of the approach.
Section 3 presents a probabilistic decomposition of the proposed architecture. With some fairly standard assumptions and simplifications, the loss in Eq. 3 becomes rather straightforward (weighted cross entropy)
The actual network architecture described in Section 3.2 takes a standard U-Net as a starting point and modifies it in a fairly targeted way for the different expected types of annotations. These annotations (Section 3.3) are cheaper than full labels on a same-size dataset; it is not completely clear, however, if the mentioned scribbles need to capture each instance in the training set, or if some can also be left out. Without this being explicitly mentioned, I will assume the former.
The experimental evaluation is done reasonably well, although I am not familiar with any of the presented data sets. The SES set seems to be specific to the submission, while the H&E data set has been used at least one other relevant publication (Zhang et al.). My main issue here is that at least on the SES set, which does not seem to be that large, the score difference is not that big, so dataset bias could play some part (which is unproven, but so is the opposite).
Experimental evaluation does not leave the low-number-of-classes regime, and I’m left wondering how the method might compare on a semantically much richer data set, e.g. Cityscapes. Finally, unmodified U-Net is by now a rather venerable baseline, so I’m also wondering how the proposed multi-task learning could be used in other (more recent) architectures, i.e. whether the idea can be generalized sufficiently.
While I think the ideas per se have relatively minor novelty, the combination seems novel to me, and that might warrant publication.


The geometric properties of loss surfaces, such as the local flatness of a solution, are associated with generalization in deep learning. The Hessian is often used to understand these geometric properties. We investigate the differences between the eigenvalues of the neural network Hessian evaluated over the empirical dataset, the Empirical Hessian, and the eigenvalues of the Hessian under the data generating distribution, which we term the True Hessian. Under mild assumptions, we use random matrix theory to show that the True Hessian has eigenvalues of smaller absolute value than the Empirical Hessian. We support these results for different SGD schedules on both a 110-Layer ResNet and VGG-16. To perform these experiments we propose a framework for spectral visualization, based on GPU accelerated stochastic Lanczos quadrature. This approach is an order of magnitude faster than state-of-the-art methods for spectral visualization, and can be generically used to investigate the spectral properties of matrices in deep learning.

This paper analyzes the spectrum of the Hessian matrix of large neural networks, both from the theoretical and the empirical perspective. The main contributions are (1) a theoretical analysis of the max/min eigenvalues, showing that the max/min are larger/smaller for empirical Hessians compared to "true" Hessians, and (2) an empirical analysis and visualization of spectra using a Lanczos quadrature approach for a variety of training methods and architectures.
Generally, analyzing the local curvature of (approximate) optima is an important problem for building a better understanding of neural network loss surfaces and for shedding light on how that geometry might be related to generalization performance. Therefore, I find the topic of this paper salient, and the questions analyzed are sure to be of interest to a broad range of theoretically-minded and practically-minded deep learning researchers.
However, I did not find the theoretical analysis to be especially novel, deep, or informative. I also did not find the conclusions reached to be particularly compelling, especially insofar as they related to how the geometry might affect generalization performance. For these reasons, I think this paper is a bit below the bar for acceptance, and I'd encourage the authors to add more content and depth to their analysis in a future submission.
Let me elaborate on my above comments in slightly more detail.
On the Theoretical analysis:
The general observation being made here is that adding more data will tighten the spectrum. This behavior is well-known, e.g. simply adjusting the aspect ratio of the Marcenko-Pastur distribution gives the same effect. Similar behavior is known in more complicated models. The result in Theorem 1 is not especially novel and is probably available in the literature in some form (I didn't find an immediate reference, but even without one, and even if the precise result isn't presented formally in prior work, this type of result is quite well-known).
Some of the assumptions are very strong, especially the independence of the entries in \epsilon. While I actually suspect that the overall conclusion of spectral broadening may be fairly robust, the argument in section 4.3 did not convince me. There could be very strong correlations in \epsilon that get induced by the architecture and learned weights, and these correlations could become larger as the dataset size grows, leading to big discrepancies with the predictions. This would probably be most pronounced when transitioning from the under-parameterized to over-parameterized regimes.
On the Empirical analysis:
As the authors note, the main Lanczos quadrature method being utilized was introduced in several prior works, and the "improvement" offered here is basically a small tweak (reducing the number of random vectors). The argument in section 5.2 for reducing this number to one was not particularly compelling, and if the author's regarded this modification as an important contribution, I would have appreciated a more thorough variance analysis, both from the theoretical and empirical perspectives.
Additionally, related to error analysis, the method utilized in this work (and in the prior works that this work build upon) does not really have a robust way of reliably measuring the error in the spectral density estimation. There is no guarantee that the estimate will be accurate. Such an error estimate should depend on the ground-truth spectrum, and if the ground-truth spectrum is particularly pathological (say, having a large density near zero and a sizable number of large outliers  [which may in fact be the case that may happen in practice!]), the estimate may be quite bad. So I'm actually not completely convinced that the results presented here are accurate estimates of the actual spectrum.
Finally, I found the method for generating the "True Hessian" to be quite ad-hoc and likely severely biased. To what extent are the results skewed by the particular data augmentation procedure being utilized? Any effort to control for this kind of bias would be an important addition to this paper. Additionally, it might have been useful to corroborate some of the main claims with regard to the True Hessian (vs Empirical Hessian) by utilizing a synthetic data distribution from which unbiased samples could be drawn. Lastly, even though the number of augmented samples was large, it was not very large compared to the number of parameters. Perhaps studying a regime in which the ratio was farther from one would have been useful.
On the conclusions:
One of the main important motivations for this analysis is to understand how curvature might relate to generalization. This motivation was put forward in the introduction and was the basis for the experiments comparing the Normal and Overfit model classes. However, I found this single type of comparison to provide very weak evidence in favor of the conclusion that flat minima generalize better. A more compelling argument in this direction would have included substantially more empirical evidence, or (perhaps in better keeping with the main perspectives of this paper) additional theoretical analysis of this particular point.
_____________
Overall, I think the problem studied in this paper is quite interesting, but I did not find the theoretical or empirical contributions to be sufficiently novel, deep, or informative to merit publication in their present form.


In information extraction, event extraction is one of the types that extract the specific knowledge of certain incidents from texts. Event extraction has been done on different languages texts but not on one of the Semitic language Amharic. In this study, we present a system that extracts an event from unstructured Amharic text. The system has designed by the integration of supervised machine learning and rule-based approaches together. We call it a hybrid system. The model from the supervised machine learning detects events from the text, then, handcrafted rules and the rule-based rules extract the event from the text. The hybrid system has compared with the standalone rule-based method that is well known for event extraction. The study has shown that the hybrid system has outperformed the standalone rule-based method. For the event extraction, we have been extracting event arguments. Event arguments identify event triggering words or phrases that clearly express the occurrence of the event. The event argument attributes can be verbs, nouns, occasionally adjectives such as˜rg/wedding and time as well.

The paper studies extracting events from unstructured text, specifically on the low resource language Amharic. The paper proposes to combine rule-based based with a learning-based approach.
Strength
-	Looks at the low resource language Amharic and mines a large corpus of text.
-	The paper compares 3 learning algorithms, NB, Libsvm, J48. As well as a hybrid versus a rule based approach.
Weaknesses:
1.	Prior work
1.1.	The paper misses to discuss prior work on Amahric copora
Rychlý P., Suchomel V. (2016) Annotated Amharic Corpora. In: Sojka P., Horák A., Kopeček I., Pala K. (eds) Text, Speech, and Dialogue. TSD 2016. Lecture Notes in Computer Science, vol 9924. Springer, Cham
2.	Experimental evaluation
2.1.	Missing ablations
2.1.1.	The paper claims “Amharic presents sophisticated language-specific issues” but does not evaluate how the proposed approach handles those specific challenges.
2.1.2.	The authors “normalize” characters with the same meaning “to an ordinary conceivable form” which “helps the performance of [their] system”; however, the paper does not ablate this benefit.
2.2.	The authors “applied k-fold cross- validation which split our dataset randomly into k groups”. It would be great if the authors report the standard deviation across the results to understand the significant of differences.
2.3.	The paper misses to show any qualitative results showing how the system works and which error it makes.
2.4.	The paper notes that the “Amharic has a subject-object verb agreement and other morphological feature makes cumbersome the construction of rule” as well as the large unsupervised corpus they have available suggest to me that the authors should explore unsupervised pre-training a transformer-based network, such as BERT or other approaches which have successfully been used on other languages.
3.	Clarity: The paper’s clarity could be improved:
3.1.	On the one hand by clearly stating the contributions and by improving the clarity of the writing throughout.
3.2.	It is not clear how Table 2 relates Table 1. The authors note it is the same dataset. Does Table 2 show Weighted Ave of On and Off-event?
3.3.	What is the large unlabeled corpus used for?
Minor:
- considering using the correct citation style: “27 million speakers Mulugeta and Gasser (2012) “ -> 27 million speakers (Mulugeta and Gasser, 2012)
- don’t capitilzie after “;”:
- duplication of very similar sentences: “Events in Amharic text are predominantly expressed through verbs and nouns, but, the linguistic structure and morphological richness has not represented in models that have been developed for other languages. Events in Amharic are ex- pressed predominantly through verbs and nominal, but, the linguistic structure and morphological richness highly matter to apply models used for other languages.”
Conclusion:
While it is interesting and valuable that this paper studies a low-resource language. The paper misses comparisons to prior work for event extraction and misses to include ablations which show the value of the introduced rules and design decisions made.
=== post author response ====
I thank the authors for their response.
Unfortunately, the authors did not use the chance to update the pdf to improve the clarity (3.); report standard deviation (2.2) to judge significance, although they should have this data; nor did they include a revised discussion of prior work.
As clarity and experimental evaluation are also major concerns of the other reviewers and it is unclear if and how they will be addressed in a revision I do not recommend accepting the paper.


Reinforcement learning agents that operate in diverse and complex environments can benefit from the structured decomposition of their behavior. Often, this is addressed in the context of hierarchical reinforcement learning, where the aim is to decompose a policy into lower-level primitives or options, and a higher-level meta-policy that triggers the appropriate behaviors for a given situation. However, the meta-policy must still produce appropriate decisions in all states. In this work, we propose a policy design that decomposes into primitives, similarly to hierarchical reinforcement learning, but without a high-level meta-policy. Instead, each primitive can decide for themselves whether they wish to act in the current state. We use an information-theoretic mechanism for enabling this decentralized decision: each primitive chooses how much information it needs about the current state to make a decision and the primitive that requests the most information about the current state acts in the world. The primitives are regularized to use as little information as possible, which leads to natural competition and specialization. We experimentally demonstrate that this policy architecture improves over both flat and hierarchical policies in terms of generalization.

The paper draws upon the idea of information bottleneck to do task decomposition so as to learn policy primitives similar to hierarchical reinforcement learning that combine together in a competitive manner to specialize in different parts of the task's domain. These policy primitives don't need a higher-level meta-policy to stitch them together. Instead the decision is made in a decentralized manner balancing the cost of information acquisition with maximizing rewards.
The paper seems to build on the idea of decomposing the task primitives that specialize in different parts of the state space. A related recent paper [1] which used similar ideas, albeit with a central coordinator to do task composition is missing from related works.
There are other issues with the paper as well.
- Decision making is not exactly "decentralized"? Computing $Z$ in $\alpha_k$, still requires the values from other primitives?
- Sec 5.1 seems tacked on. Motion Imitation is not RL. Experiment details about motion imitation unclear from description.
- In Figure 5: Zero-shot generalization, it's unclear from the plot whether it actually generalized to solve the tasks A and B. Relative frequency of activation are meaningless without reporting the actual performance on these environments. Where do the 4 indices come from? From the plot above there were only 2 indices but plot below has 4.
- Continuous control tasks required pretraining.
Overall the experiments seem a little underwhelming. More details on transfer performance without pretraining would be quite helpful. I like the idea of a competitive ensemble figuring out a useful task decomposition and using an information bottleneck like approach makes sense.
[1] https://dl.acm.org/citation.cfm?id=3331671
*Edit*: Updated the score after the reading the revision and the authors' responses.


Combined with policy and value neural networks, Monte Carlos Tree Search (MCTS) is a critical component of the recent success of AI agents in learning to play board games like Chess and Go (Silver et al., 2017). However, the theoretical foundations of MCTS with policy and value networks remains open. Inspired by MCTS, we propose A MCTS, a novel search algorithm that uses both the policy and value predictors to guide search and enjoys theoretical guarantees. Specifically, assuming that value and policy networks give reasonably accurate signals of the values of each state and action, the sample complexity (number of calls to the value network) to estimate the value of the current state, as well as the optimal one-step action to take from the current state, can be bounded. We apply our theoretical framework to different models for the noise distribution of the policy and value network as well as the distribution of rewards, and show that for these general models, the sample complexity is polynomial in D, where D is the depth of the search tree. Empirically, our method outperforms MCTS in these models.

This paper proposes A*MCTS, which combines A* and MCTS with policy and value networks to prioritize the next state to be explored. It further establishes the sample complexity to determine optimal actions. Experimental results validate the theoretical analysis and demonstrate the effectiveness of A*MCTS over benchmark MCTS algorithms with value and policy networks.
Pros:
This paper presents the first study of tree search for optimal actions in the presence of pretrained value and policy networks. And it combines A* search with MCTS to improve the performance over the traditional MCTS approaches based on UCT or PUCT tree policies. Experimental results show that the proposed algorithm outperform the MCTS algorithms.
Cons:
However, there are several issues that should be addressed including the presentation of the paper:
•	The algorithm seeks to combine A* search with MCTS (combined with policy and value networks), and is shown to outperform the baseline MCTS method. However, it does not clearly explain the key insights of why it could perform better. For example, what kind of additional benefit will it bring when integrating the priority queue into the MCTS algorithms? How could it improve over the traditional tree policy (e.g., UCT) for the selection step in MCTS? These discussions are critical to understand the merit of the proposed algorithms. In addition, more experimental analysis should also be presented to support why such a combination is the key contribution to the performance gain.
•	Many design choices for the algorithms are not clearly explained. For example, in line 8 of Algorithm 2, why only the top 3 child nodes are added to the queue?
•	The complexity bound in Theorem 1 is hard to understand. It does not give the explicit relations of the sample complexity with respect to different quantities in the algorithms. In particular, the probability in the second term of Theorem 1 is hard to parse. The authors need to give more discussion and explanation about it. This is also the case for Theorems 2-4. The authors give some concrete examples in Section 6.2 for these bounds. However, it would be better to have some discussion earlier right after these theorems are presented.
•	The experimental results are carried out under the very simplified settings for both the proposed algorithm and the baseline MCTS. In fact, it is performed under the exact assumption where the theoretical analysis is done for the A*MCTS. This may bring some advantage for the proposed algorithm. It is not clear whether such assumptions hold for practical problems. More convincing experimental comparison should be done under real environment such as Atari games (by using the simulator as the environment model as shown in [Guo et al 2014] “Deep learning for real-time atari game play using offline monte-carlo tree search planning”).

Other comments:
•	It is assumed that the noise of value and policy network is zero at the leaf node. In practice, this is not true because even at the leaf node the value could still be estimated by an inaccurate value network (e.g., AlphaGo or AlphaZero). How would this affect the results?
•	In fact, the proof of the theorems could be moved to appendices.
•	In the first paragraph of Section 6.2, there is a typo: V*=V_{l*}=\eta should be V*-V_{l*}=\eta ?


We propose Axial Transformers, a self-attention-based autoregressive model for images and other data organized as high dimensional tensors. Existing autoregressive models either suffer from excessively large computational resource requirements for high dimensional data, or make compromises in terms of distribution expressiveness or ease of implementation in order to decrease resource requirements. Our architecture, by contrast, maintains both full expressiveness over joint distributions over data and ease of implementation with standard deep learning frameworks, while requiring reasonable memory and computation and achieving state-of-the-art results on standard generative modeling benchmarks. Our models are based on axial attention, a simple generalization of self-attention that naturally aligns with the multiple dimensions of the tensors in both the encoding and the decoding settings. Notably the proposed structure of the layers allows for the vast majority of the context to be computed in parallel during decoding without introducing any independence assumptions. This semi-parallel structure goes a long way to making decoding from even a very large Axial Transformer broadly applicable. We demonstrate state-of-the-art results for the Axial Transformer on the ImageNet-32 and ImageNet-64 image benchmarks as well as on the BAIR Robotic Pushing video benchmark. We open source the implementation of Axial Transformers.

This paper proposes axial attention as an alternative of self-attention for data arranged as large multidimensional tensors, which costs too much computational resource since the complexity of traditional self-attention is quadratic in order to capture long-range dependencies for full receptive fields. The axial attention is applied within each axis of the data separately while keeping information along other axes independent. Therefore, for a d-dimensional tensor with N = S^d, axial attention saves O(N^{(d−1)/d}) computation over standard attention. The proposed axial attention can be used within standard Transformer layers in a straightforward manner to produce Axial Transformer layers, without changing the basic building blocks of traditional Transformer architecture.  The authors did experiments on two standard datasets for generative image and video models: down-sampled ImageNet and BAIR Robot Pushing, and they claim that their proposed method matches or outperforms the state-of-the-art on ImageNet-32 and ImageNet-64 image benchmarks and sets a significant new state-of-the-art on the BAIR Robot Pushing video benchmark.
Reasons to accept:
1.	Simple, easy-to-implement yet effective approach to adapt self-attention to large multidimensional data, which can save considerable computation for efficiency, while still have competitive performance.
2.	Clear writing, with sufficient but not redundant introduction of background knowledge and explanation of both the advantages and drawbacks of existing models (too large computational complexity on high-dimensional data).
Suggestions for improvement:
1.	It would be better if the authors can provide more analysis or case study to show the reason why Axial attention (Axial Transformer) can reach good performance even if it omits considerable operations compared to traditional Transformers, or to show why the attention operations within axis are important instead of attention operations between axis.
2.	Definition of “axis” should be more clear in section 3 (there could be some ambiguities of “axis”).


Batch Normalization (BatchNorm) has shown to be effective for improving and accelerating the training of deep neural networks. However, recently it has been shown that it is also vulnerable to adversarial perturbations. In this work, we aim to investigate the cause of adversarial vulnerability of the BatchNorm. We hypothesize that the use of different normalization statistics during training and inference (mini-batch statistics for training and moving average of these values at inference) is the main cause of this adversarial vulnerability in the BatchNorm layer. We empirically proved this by experiments on various neural network architectures and datasets. Furthermore, we introduce Robust Normalization (RobustNorm) and experimentally show that it is not only resilient to adversarial perturbation but also inherit the benefits of BatchNorm.

This paper addresses a limitation of BatchNorm: vulnerability to adversarial perturbations. The authors propose a possible explanation of this issue and correspondingly an alternative called RobustNorm to tackle this problem. Specifically, the authors observe that the statistics of BatchNorm for training and inference are different, resulting in different data distributions for training and inference. To solve this problem, the authors propose to use min-max rescaling instead of normalization. In addition, the running average is calculated with mean and the running mean of the denominator during inference. Experimental results show significant improvement of robustness and also comparable accuracy for clean data.
The paper is well-written and the contributions are stated clearly. The explanation of vulnerability is reasonable. The proposed solution is simple but effective.
However, I have several concerns:
*The authors verify that the running average is the main culprit of vulnerability to adversarial attack, but provide no further investigation of why this happens. A possible solution is the drift in input distributions, but the manuscript does not state clearly how is the distribution changed. Further experiments would have made this claim more convincing.
*The proposed method involves a hyper-parameter \rho, but it may result in problematic issues. The variance of input is of the same order of magnitude as (max(x)-min(x))^2. If \rho is set to other value, the magnitude of gradient will change drastically during back-propagation. Although \rho can be set to 0.2, it still seems ad-hoc. Experiments on more datasets and the sensitivity of the proposed method to \rho would have validated the claims of the authors.


The Lottery Ticket Hypothesis from  conjectures that, for typically-sized neural networks, it is possible to find small sub-networks which train faster and yield superior performance than their original counterparts. The proposed algorithm to search for "winning tickets", Iterative Magnitude Pruning, consistently finds sub-networks with 90 − 95% less parameters which train faster and better than the overparameterized models they were extracted from, creating potential applications to problems such as transfer learning. In this paper, we propose Continuous Sparsification, a new algorithm to search for winning tickets which continuously removes parameters from a network during training, and learns the sub-network's structure with gradient-based methods instead of relying on pruning strategies. We show empirically that our method is capable of finding tickets that outperforms the ones learned by Iterative Magnitude Pruning, and at the same time providing faster search, when measured in number of training epochs or wall-clock time.

To the authors of paper 2504: I have posted a private comment for the reviewer/AC discussion period based on your author responses and your revised paper. I want you to be able to see my full response, but I can't post additional public comments to the paper. As such, I'm editing my review with exactly what I sent to the AC.
Thank you to the authors for your thoughtful rebuttal and for updating the paper with both new text and (more impressively) new experiments. I read all three of your rebuttals and re-read the revised paper in detail. My comment to the AC is at the bottom of this message. I don't know if you will get an alert that the review was updated, but I hope you get a chance to take a look.
===========================================
EXECUTIVE SUMMARY OF REVIEW
Summary of paper: This paper proposes a technique that simultaneously trains a neural network and learns a pruning mask. The goal of this technique is to make it faster to retroactively find sparse subnetworks that, from a point early in training, could have trained in isolation to the same performance as the full network ("winning tickets"). The best subnetworks found by this technique outperform those found by existing techniques [1] at every sparsity level.
Summary of review: The technique introduces several new hyperparameters whose values are asserted without describing the extent of the search necessary to find them; it unclear whether the cost of the search cancels out the efficiency gains. In addition, the proposed technique is inconsistent across runs. The sparsity and accuracy of the subnetworks it produces vary greatly, and there are no hyperparameters to explicitly control these outcomes. It is also unclear whether results vary from run to run even with the same hyperparameters.  As such, this technique is not clearly a cost reduction as compared to existing approaches in [1] when it comes to studying lottery tickets. Moreover, the paper implicitly proposes a new pruning technique, and it should be evaluated against other related techniques in the pruning literature. Finally, the evaluation needs more rigor.
Conclusion: It is difficult to determine whether the technique is an improvement over existing methods since the true costs of using it in practical workflows are unclear. Weak reject.
Opportunity to improve score: Include a more detailed analysis of the overall costs of finding winning tickets at a target sparsity using the proposed technique, particularly including the costs of hyperparameter search necessary to do so. Include multiple replicates of experiments, experiments on more networks, and information about the variance of performance across runs with the same hyperparameters. Compare against other state-of-the-art pruning techniques.
PROBLEM STATEMENT AND PROPOSED SOLUTION
Problem: Winning tickets are currently expensive to find. The best known procedure [1, 2] is "iterative magnitude pruning" (IMP), which involves repeatedly training a network to completion, pruning by a fixed percentage, and "rewinding" weights to an early iteration of training until the network reaches the desired level of sparsity. To reach sufficient sparsity on standard networks, this procedure must be repeated 10 or more times.
Goal: To propose a procedure that finds winning tickets more efficiently.
Significance: A more efficient procedure would make it easier to study the lottery ticket phenomenon. (Whether studying that phenomenon is, itself, significant is debatable.) Personally, I have extensive experience using IMP to find winning tickets, so techniques to reduce the cost of finding winning tickets would be very valuable for my work.
Proposed solution: The authors propose "continuous sparsification" (CS), which makes it possible to learn which weights to prune simultaneously with the weights themselves. Accompanying each parameter w in the network is a second parameter s. The actual weight used in the network is w * sigmoid(s * beta), where beta is a temperature hyperparameter. If the learned value of s is such that sigmoid(s * beta) is approximately 0 at the end of training, then the parameter has been pruned. The value of beta increases exponentially throughout training, meaning the output of the sigmoid will be closer to a hard 0 or 1, producing a pruning mask. To ensure sparsification happens, a regularization term lambda |sigmoid(beta * s)| is added. When run over multiple iterations, values of s are reset to their original values for weights that are not pruned. In addition, weights are either rewound (as in IMP) or left at their final values (as in [3]).
Novelty: The paper is slightly novel. The technique is a variation of those proposed in [4] and [5], but the changes are meaningful. This is the first known use for finding winning tickets, but any pruning technique could hypothetically be used for this purpose. In effect, the paper proposes a new pruning technique that is primarily evaluated for its efficacy in finding winning tickets.
TECHNICAL REVIEW
* This technique introduces several new hyperparameters: lambda, initial and final betas, and the initial values for s. The paper suggests good values for these hyperparameters for both networks considered. These values were presumably found through hyperparameter search of some kind. How extensive was this hyperparameter search, and what is the range of "good" combinations of values? This is not just a methodological footnote; this paper's stated goal is to improve the efficiency of finding winning lottery tickets, and - if good hyperparameters are hard to find - then that defeats the purpose of a more efficient technique. IMP, while far less efficient epoch for epoch as compared to CS, requires no hyperparameter search; global pruning 20% of parameters per iteration seems to work well in general [2]. In a revised version of the paper, I would be eager to learn more about this set of tradeoffs, since that is what matters in practice.
* The authors only study two networks: a toy convolutional network and a small Resnet. It is hard to draw broad conclusions from such a limited set of examples. I would be particularly interested in seeing how this technique performs on a large-scale network for ImageNet (e.g., Resnet-50), since these are the situations where IMP becomes particularly cost-prohibitive. If the technique works well in these settings, it would enable lottery ticket research at much larger scales than is currently possible. If the technique works as efficiently at this scale, then doing could even be feasible during the rebuttal period. (I acknowledge getting experiments working on ImageNet is no small undertaking in terms of both engineering time and cost, but it would improve my confidence to see those results.)
* The authors did an admirably careful job replicating the networks in [1], which include a variety of nonstandard hyperparameters.
* Did the paper study Resnet-18 (a network designed for ImageNet with 11.2M parameters) or Resnet-20 (a network designed for CIFAR-10 with 272K parameters)? Frankle et al. [1, 2] describe Resnet-20 in their appendices but mistakenly refer to it as Resnet-18 throughout both papers, so I wanted to clarify. Based on the final test accuracy of the network, it appears to be Resnet-20; if so, I'd urge you to call it as such and note in a parenthetical or footnote that it's the same network as in [1, 2] but with Frankle et al's mistaken name corrected.
* Are the values of s0 for CS sampled from a distribution, or are they all the same, fixed value?
* Do the extra parameters lead to longer wall-clock training times? If you are making an argument about a more efficient technique, this is an important consideration.
* Figure 2 includes the same graph twice. I believe a different graph should appear on the left, and I am eager to take a look at it in a revised version of the paper.
* It does not appear that multiple replicates of each experiment were run with different network initializations in Figure 3. There don't appear to be any error bars on Figure 3, suggesting that this only represents a single initialization. Considering the wide variance in performance achieved by continuous sparsification across runs in Figure 3 (right), this graph needs to include multiple runs and error bars. (I assume the multiple runs shown in Figure 3 (right) are with different hyperparameters?)
* What is the performance of CS as a pruning technique? By finding winning tickets, CS is also implicitly pruning the network. Is this competitive with L0 Regularization [4]? Is it more efficient than iterative magnitude pruning as in [3]? If this is a more efficient technique for finding winning tickets, it is also liable to be a more efficient pruning method, which seems like even broader impact for the proposed technique. Alternatively, if this technique is less effective than comparable work (especially [4]) as a pruning method, then it is possible comparable work (especially [4]) might also produce better winning tickets, in which case the importance of this work is diminished. In an updated version of the paper, I would be interested in seeing an evaluation of CS as a pruning technique independent of the lottery ticket hypothesis (and compared to standard techniques in the pruning literature as such). I have a hard time seeing any reason why new techniques for finding lottery tickets are any different than new pruning techniques, and they should be evaluated as such in the context of the broader literature on pruning.
* The last paragraph of Section 4.1 is missing important details that are necessary to evaluate the utility of continuous sparsification in comparison to IMP. "In only two iterations, CS finds a ticket with over 77% sparsity..." - for which hyperparameters, and how many hyperparameters had to be explored to find these values? How was the pareto curve obtained? How many different runs were necessary to create it? How many total epochs of training did it take to find that curve? If obtaining this pareto curve required many runs of CS, then it may not be any more efficient than running IMP in practice. The pareto curves appear to make CS look misleadingly effective, since they hide many of the actual costs involved (e.g., hyperparameter search, number of separate runs that were conducted to produce the curve, etc.). Greater transparency of this aspect of the paper would go a long way toward increasing my confidence that the findings are an improvement over IMP.
* The sparsity and accuracy of subnetworks found by CS appears to vary widely from run to run as shown in Figure 3 (right). The final sparsity appears to be a function of the values of s0, lambda, and beta, potentially along with luck from the optimization process. For the same values of s, lambda, and beta, how widely do the final sparsities and accuracies vary? If I wanted to find a winning ticket with a particular sparsity using CS, what would the procedure look like for doing so? Would I have to sweep across values of these hyperparameters, or is there a more straightforward way to do so? The practical usefulness of the procedure hinges on these questions.
* "We associate the performance drop of highly sparse tickets found by our method from the second iteration onwards to the lack of weight rewinding." Why didn't you also try it with weight rewinding? That seems like an easy way to evaluate this hypothesis. For both networks, it would be interesting to see the performance of CS with and without rewinding (analogous to Appendix B in [1]).
* In the literature so far, the only winning tickets to be examined are those from IMP [1, 2]. CS is a different technique, and it likely finds different winning tickets. Do these winning tickets have different properties from [1, 2]? Can they be reinitialized? Can they be rewound earlier? These comparisons seem like an interesting scientific opportunity.
* There are a few additional comparisons that I think are vital to include in the paper to appropriately contextualize results.  They're in the bullets below.
* First comparison: it would be useful to include random reinitialization or random pruning baselines as in [1, 2] simply to make it easier for the reader to contextualize the performance of other sparse subnetworks.
* Second comparison: what happens if you run IMP such that each iteration prunes to the same sparsity as achieved by each iteration of CS? Perhaps pruning by a fixed amount per iteration in IMP is wasteful, and one can prune more aggressively during earlier iterations as CS naturally appears to do. In other words, one way of explaining the advantage of CS would be that it prunes more aggressively. Is this indeed the case? I would be very curious to know.
* Third comparison: What are the range of results achieved if IMP and CS are run "one-shot" (pruning after just one iteration; in the case of IMP, pruning directly to a desired sparsity)? That is, how well can these techniques do with just a single iteration?
WRITING
The writing is excellent. The prose is clear, and I was able to fully understand a relatively sophisticated technique on the first read through the paper. Writing of this quality is rare, and the authors should be commended for it.
OVERALL: Weak Reject
The problem statement is that IMP is not efficient. The paper claims that CS is more efficient. However, the paper does not present a convincing case that CS is, on the whole, more efficient when taking into account hyperparameter search to get CS to work, hyperparameter search to target a particular sparsity, potential variance across runs of CS, potential additional training costs of CS, and the possibility that IMP might be able to work comparable well given a more aggressive pruning schedule.
In addition, the evaluation needs more experiments, including multiple replicates for each experiment and more networks (ideally one on ImageNet).
Finally, I am unclear on what distinguishes CS from any other pruning technique, and it should be evaluated in the context of the broader pruning literature.
If the authors clarify that the overall cost of CS (including all of the factors listed above) is lower than for IMP, address technical concerns about the evaluation, and evaluate CS as a pruning technique in an updated version of the paper, I will update my score accordingly.
[1] Frankle & Carbin. "The Lottery Ticket Hypothesis." ICLR 2019.
[2] Frankle et al. "Stabilizing the Lottery Ticket Hypothesis." Arxiv.
[3] Han et al. "Learning both Weights and Connections for Efficient Neural Networks." NeurIPS 2015.
[4] Louizos et al. "Learning Sparse Neural Networks through L0 Regularization." ICLR 2018
[5] Zhou et al. "Deconstructing Lottery Tickets: Signs, Zeros, and the SuperMask." NeurIPS 2019.
=====================================
COMMENT TO THE AC AFTER READING REBUTTALS AND REVISED PAPER
TLDR
I believe the technique is novel and should be published, regardless of whether it is actually more efficient than IMP for practical workflows.
However, I believe the current evaluation is inadequate, both in the experiments and in the way data is presented. Namely, it is impossible to actually compare the costs of CS and IMP in the scenarios the authors evaluate, despite the fact that efficiency is the authors' claimed contribution. From the data as presented, it is unclear whether CS is actually more efficient than IMP for scientific use-cases. I'm not particularly concerned if there isn't an efficiency advantage - it's an innovative contribution regardless. My concern is that it is impossible to compare the costs of these techniques given the current presentation of data. That speaks to flaws in the evaluation section.
I therefore maintain my score: weak reject. The technique deserves to be published, but the paper in its current form does not.
INTRODUCTION
After reading the original submission, I had the following questions:
1) Right now, the only way to find winning lottery tickets is through training the network and pruning it. IMP can be instantiated with any pruning technique, and the authors are really proposing a new pruning technique for use in the IMP framework. How does continuous sparsification perform as a pruning technique for network compression (i.e., independent of the lottery ticket hypothesis)?
2) How efficient is continuous sparsification (CS) in the scientific use cases where one would seek to find winning lottery tickets? In my research experience, there are two such use cases: (a) producing a winning lottery ticket with a specific sparsity and (b) producing winning lottery tickets across the full range of different sparsities.
In their reubttal, the authors addressed these underlying questions:
AS A PRUNING TECHNIQUE
"We have added new experiments showing that our method yields competitive results when pruning VGG trained on CIFAR, outperforming both magnitude pruning and stochastic l0 regularization"
My response: The initial results that the authors present are quite impressive on a VGG-style network for CIFAR-10. However, I find it concerning that the authors study pruning on this specific model but no others, particularly because they focus on a different network (namely, Resnet-20) in all other experiments in the paper. Since the authors are, in essence, proposing a new pruning technique, I would like to see it comprehensively evaluated as such on a range of networks against other baselines (as the authors do in Figure 4 for one network - those baselines are fine to me).
EFFICIENCY OF CONTINUOUS SPARSIFICATION
- It increases the cost of each individual network training run slightly: "Continuous Sparsification on a 1080 Ti: our method resulted in 15% extra wall-clock time per training epoch."
- PRODUCING WINNING TICKETS ACROSS THE FULL RANGE OF SPARSITIES: Throughout the paper, the authors present "pareto curves" showing the highest accuracy achieved by CS subnetworks at various sparsities. I find these pareto curves misleading: they hide the fact that CS had to be run several times with different hyperparameters to produce these curves.
For example, in Figure 6 (right), the authors produce the pareto curves by training Resnet-20 (by my count) 22 times (each point along the purple lines). The corresponding IMP curve appears to have 14 points. In other words, to get winning lottery tickets across all sparsities, CS must be run for more iterations than IMP - it appears to be less efficient, contradicting the authors' core claim. The best argument in favor of CS is that one could perform each of these runs in parallel if sufficient GPUs were available, meaning less wall-clock time would be required.
One caveat to this analysis: the authors state that they run CS "without rewinding," meaning that the second iteration of CS requires less training time than fully training the network (as IMP requires). The authors do not state how long they train when they *aren't* rewinding, so it is impossible to compare the efficiency of CS with IMP. All they say is that it "allow[s] for even faster ticket search." They also do not study IMP without rewinding, which would be a helpful baseline for comparing to CS without rewinding.
***In short, if the authors are making an argument that one technique is more efficient than the other on an epoch-for-epoch basis, they need to actually plot the epochs required by each technique.***
- PRODUCING A WINNING TICKET AT A SPECIFIC SPARSITY: My concern about this use case is that there is a non-intuitive relationship between the initialization of the sparsity parameters and the final sparsity of the network. As the authors state in the rebuttal: "To achieve a desired sparsity, one can either perform runs in parallel with different values for s_0, or perform sequential binary search if the goal is to minimize the overall computational cost and not wall-clock time." In other words, there is no precise way to target a particular sparsity other than trying many hyperparameter configurations. The authors do not provide any concrete costs of using CS in this way in comparison to IMP, and - from a usability perspective - this is a challenging workflow.
OTHER CONCERNS
- The authors only study CS on one toy network (the six-layer convolutional network, which - in my experience - is a particularly easy setting compared to deeper networks) and one "real" network (Resnet-20 on CIFAR-10). I would like to see results on other networks for CIFAR-10 (for example, the VGG network in Section 4.3).
- More importantly, I would like to see results on an ImageNet network. If CS makes finding winning lottery tickets more efficient as the authors claim, then finding winning tickets efficiently on an ImageNet network should be an excellent demonstration of their contribution. This scenario has stretched IMP to its breaking point, as the authors note.
ARGUMENTS IN FAVOR OF ACCEPTING
+ The authors propose a new pruning technique that appears to improve upon the increasingly popular L0-regularization technique.
+ With the right hyperparameters, the proposed technique makes it possible to find winning lottery tickets more efficiently than existing methods (i.e., IMP with magnitude pruning).
+ The winning tickets found by the proposed technique reach higher accuracy than IMP winning tickets and produce winning tickets at more extreme sparsities, improving upon our knowledge of the existence of winning lottery tickets.
ARGUMENTS IN FAVOR OF REJECTING
- The authors perform only minimal evaluation of their method as a pruning technique. It is possible that this is a missed opportunity to show an additional contribution. It is also possible that other, existing pruning techniques outperform CS at both pruning and finding winning lottery tickets.
- The technique is hard to use for the two existing use cases for finding winning tickets. In particular, there is no way to search for winning tickets at a specific sparsity.
- It is unclear whether CS is actually more efficient than IMP on an epoch-for-epoch basis, even though this is the main claimed contribution. The authors do not disclose - let alone plot - the number of training epochs required to find (a) winning tickets at a specific sparsity and (b) winning tickets at a range of sparsities, so it is impossible to make these comparisons. Meanwhile, the pareto curves the authors present are misleading, since they are the amalgamation of many separate runs of CS.
- The authors study their technique on only one "real" network (Resnet-20) for finding winning tickets and a separate "real" network (VGG-19) for pruning. The authors do not show how CS performs in other challenging settings, especially ImageNet.
- This work is only valuable if we believe that "lottery ticket hypothesis" work is valuable. In other words, this is a narrow contribution to an already-narrow area of study. This is one reason why pitching CS as a pruning technique would make this a stronger paper. I personally believe that "lottery ticket" work is valuable area of study, but I understand that it may not be seen as such in the broader ICLR community.
CONCLUSION
I believe the technique deserves to be published regardless of whether it is actually more efficient than IMP. It is a novel contribution to both our knowledge of pruning and of finding winning lottery tickets. The paper is exceptionally well written, and - as such - I believe it will help to inspire other research in this area.
However, I do not believe that the current paper - namely, the current evaluation - should be published. The paper presents no concrete data on the comparative costs of performing CS and IMP even though the core claim is that CS is more efficient. The paper does not disclose enough detail to compute these costs, and it seems like CS is more expensive than IMP for standard workflows. Moreover, the current presentation of the data through "pareto curves" is misleadingly favorable to CS.
I also believe that the paper needs experiments on ImageNet and needs a more thorough evaluation as a pruning technique beyond the lottery ticket hypothesis.
I therefore retain my current score of "weak reject," though I am eager to hear the thoughts of other reviewers, and I am open to changing my score.


"Overlearning" means that a model trained for a seemingly simple objective implicitly learns to recognize attributes and concepts that are (1) not part of the learning objective, and (2) sensitive from a privacy or bias perspective. For example, a binary gender classifier of facial images also learns to recognize races-even races that are not represented in the training data-and identities.
We demonstrate overlearning in several vision and NLP models and analyze its harmful consequences. First, inference-time representations of an overlearned model reveal sensitive attributes of the input, breaking privacy protections such as model partitioning. Second, an overlearned model can be "re-purposed" for a different, privacy-violating task even in the absence of the original training data.
We show that overlearning is intrinsic for some tasks and cannot be prevented by censoring unwanted attributes. Finally, we investigate where, when, and why overlearning happens during model training.

This paper introduces the problem of overlearning, which can be thought of as unintended transfer learning from a (victim) source model to a target task that the source model’s creator had not intended its model to be used for. The paper raises good points about privacy legislation limitations due to the fact that overlearning makes it impossible to foresee future uses of a given dataset.
Background on censoring is well developed, and helps position the submission. However, the relation to transfer learning is not sufficiently outlined in the introduction. In some ways, overlearning is a form of unintended transfer learning. In particular, the connection appears explicit in the case of model repurposing (Section 3.2). Editing the introduction to tease apart this relationship to transfer learning would help readers forge an intuition for what the paper considers.
While the de-censoring algorithm is intuitive, it is not clear what assumptions are being made because of ambiguous notation in Algorithm 1. How does Algorithm 1 train E_aux and C_aux? Does that step assume the adversary has knowledge of the algorithm use to train E and C? Descriptions of the experimental setup from Section 4.2 seem to indicate that this is the case. This is not necessarily an issue if the proposed attack is demonstrating a limitation of learning rather than a practical attack.
Experiments overall show that censoring does not mitigate overlearning in a way that is robust to de-censoring. One aspect of the experiments that is unclear is the auxiliary dataset: what is the auxiliary dataset used by the adversary for each of the datasets considered in experiments?
Question: how does simultaneous censoring of all layers affect overlearning and repurposing?
Question: what is the connection between learning more complex representations and overlearning? Is the intuition that if the representation is more complex, it is more likely to contain features useful to identify the sensitive attribute?
The paper is overall well-written. Some parts of the paper omit important details (perhaps due to space constraints?), but an editorial pass should address most of these. Detailed feedback:
1 / Explaining what model partitioning means in this context would help make the introduction more self-contained.
1 / Is overlearning specific to attributes that raise fairness issues? Or is this phenomenon more general? The “Censoring representations” paragraph on page 2 seems to indicate that the phenomenon is more general.
5/ Is accuracy the best metric to report (e.g., for race attribute prediction) given that the random guessing figure suggests the data is not balanced across attribute values?
6/ Why is the effect not monotonic in Table 4? Were multiple runs averaged?


Sequence generation models are commonly refined with reinforcement learning over user-defined metrics. However, high gradient variance hinders the practical use of this method. To stabilize this method, we adapt to contextual generation of categorical sequences a policy gradient estimator, which evaluates a set of correlated Monte Carlo (MC) rollouts for variance control. Due to the correlation, the number of unique rollouts is random and adaptive to model uncertainty; those rollouts naturally become baselines for each other, and hence are combined to effectively reduce gradient variance. We also demonstrate the use of correlated MC rollouts for binary-tree softmax models which reduce the high generation cost in large vocabulary scenarios, by decomposing each categorical action into a sequence of binary actions. We evaluate our methods on both neural program synthesis and image captioning. The proposed methods yield lower gradient variance and consistent improvement over related baselines.

The paper presents a novel reinforcement learning-based algorithm for contextual sequence generation. The algorithm builds on the previously proposed MIXER algorithm and improves it by integrating gradient estimates with lower variance (augment-REINFORCE-swap-merge). To further improve the runtime complexity of the proposed algorithm, binary tree-based hierarchical softmax is applied. The algorithm is evaluated on the Karel dataset for neural program synthesis and the MS COCO dataset for image captioning.
The presentation of the paper must be improved (my score assumes that this will have been done). It is nice to have the detailed derivations when trying to dive deeper into the problem, but it hinders understanding of the main concepts at the first reading. Therefore, I would highly recommend to move most of the formulas to the appendix and keep instead only key ideas with intuitive explanations. It would also free some space in the main paper for the experiments from the appendix.
Several questions on the technical side:
1. Is there any intuition why pseudo actions tend to be equal to the true one when learning progresses? What causes this? Might it enforce any structure (like uniform)?
2. When all pseudo actions are the same, the gradient is zero. In theory, zero gradient of a function corresponds to its extremum. Does it mean that when all pseudo actions are the same, an extremum is reached or is it just an artifact of this particular estimator? Can one prove any results of this kind?
3. I understand that the ARSM estimator should be unbiased for V = 2. Does the estimator remain unbiased when V > 2?
4. In the experiments, the variance is shown to reduce significantly which is nice. However, in theory, does the ARMS guarantee non-increasing variance or can it potentially go up in some cases? If it can, have it ever been observed in practice?
5. How does the runtime of the proposed algorithm compare to the competitors?
Experiments:
1. Bunel et al (2018) report higher generalization on the Karel dataset. Is the difference due to the removal of the optional grammar checker? Can the same experiments be performed with this checker on or are there any constraints of the ARSM-based method?
2. The submitted code for the NPS experiment is actually the one by Bunel et al with their comments. I could not find any instructions or scripts reproducing the results of this paper (and I didn’t have much time to figure that out). One thing I wanted to check in the code is how the variance was computed for the plots?
Minor:
1. p. 4, “expected award” >> “expected reward”
2. g_{ARSM} defined twice in (5) and in the beginning of p. 4
3. “Fig. 1 (left two) plots” and “Fig. 1 (right two) plots” not good


In this paper, we aim to develop a novel mechanism to preserve differential privacy (DP) in adversarial learning for deep neural networks, with provable robustness to adversarial examples. We leverage the sequential composition theory in DP, to establish a new connection between DP preservation and provable robustness. To address the trade-off among model utility, privacy loss, and robustness, we design an original, differentially private, adversarial objective function, based on the postprocessing property in DP, to tighten the sensitivity of our model. An end-to-end theoretical analysis and thorough evaluations show that our mechanism notably improves the robustness of DP deep neural networks.

This paper propose an algorithm with DP preservation to train adversarially robust neural networks. To preserve DP, a single-layer linear autoencoder with shared weights is learned to extract features from training data, whose encoder is used to extract private features for the training and inference of a deeper network. To enhance robustness against various attacks, adversarial examples crafted with such attacks are injected into the training set in this algorithm. Guarantees of privacy preservation for training on both clean data and adversarial examples for the autoencoder and the inference network are given. Certified robustness of the smoothed classifier is also given, which depends on the privacy budget of each compositing mechanism. Experimental evaluations of two small (2 and 3 conv layers) networks are given on the MNIST and CIFAR10 dataset, showing improved conventional accuracy on clean samples and adversarial attacks, and certified accuracy than 4 baseline privacy-preserving algorithms.
To my knowledge, this paper provides the most comprehensive analysis so far about privacy preservation in the process of enhancing empirical adversarial robustness, as well as the impact of privacy preservation on the certified robustness of the smoothed classifier. However, the current version is quite difficult to follow for people without DP background, with some settings even conflict with other papers on adversarial robustness, and I do have some doubts about the experimental results. Specifically,
1. What is the role of adversarial examples in the proposed algorithm? Perhaps I missed something, but the authors have not shown how it is related to certified robustness in the paper. The adversarial examples seem to be used only for improving the empirical accuracy against various adversarial attacks. Empirical adversarial robustness is usually down weighted to me when certified robustness is given, therefore an algorithm enhancing only the empirical one does not seem so interesting.
2. What is even more against my intuition at first glance is that the empirical adversarial accuracy in Figure 4 is lower than the certified accuracy in Figure 6 in some situations. After a while I realized that the conventional accuracy against attacks in Figure 4,5 and the certified accuracy in Figure 6,7 are actually talking about different models, where the first one is the deterministic (or one-random-sample) inference network but the second one is for the smoothed classifier (as referred to in by Cohen et al. 2019, or the expectation of the inference network). This distinction should be addressed, since only one model can be chosen at deployment.
3. The attacks used for evaluating the empirical adversarial robustness are too weak. Only 10 steps are used. More iterative steps, e.g., 1000 or 10000 step PGD I-FGSM, should be provided to reveal the actual robustness of the networks in Figure 4,5.
4. The results of the proposed algorithm in Figure 7 is much better than the state-of-the-art, but I cannot see clearly from this paper how such improvement is achieved. With a ResNet110, [1] and [2] can only achieve around 30% certified accuracy when the maximum l_infty norm is 8/255. However, in Figure 7, the proposed algorithm is able to keep the certified accuracy above 40% at much larger perturbations, with only a 3-conv-layer network. If I am understanding the numbers correctly, could the authors explain clearly where such improvements come from?
Therefore, I tend to reject the paper before my concerns are addressed.
[1] "Filling the Soap Bubbles: Efficient Black-Box Adversarial Certification with Non-Gaussian Smoothing", https://openreview.net/forum?id=Skg8gJBFvr
[2] Salman, Hadi, et al. "Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers." NeurIPS (2019).


In this paper, we introduce a collaborative training algorithm of balanced random forests for domain adaptation tasks which can avoid the overfitting problem. In real scenarios, most domain adaptation algorithms face the challenges from noisy, insufficient training data. Moreover in open set categorization, unknown or misaligned source and target categories adds difficulty. In such cases, conventional methods suffer from overfitting and fail to successfully transfer the knowledge of the source to the target domain. To address these issues, the following two techniques are proposed. First, we introduce the optimized decision tree construction method, in which the data at each node are split into equal sizes while maximizing the information gain. Compared to the conventional random forests, it generates larger and more balanced decision trees due to the even-split constraint, which contributes to enhanced discrimination power and reduced overfitting. Second, to tackle the domain misalignment problem, we propose the domain alignment loss which penalizes uneven splits of the source and target domain data. By collaboratively optimizing the information gain of the labeled source data as well as the entropy of unlabeled target data distributions, the proposed CoBRF algorithm achieves significantly better performance than the state-of-the-art methods. The proposed algorithm is extensively evaluated in various experimental setups in challenging domain adaptation tasks with noisy and small training data as well as open set domain adaptation problems, for two backbone networks of AlexNet and ResNet-50.

Summary:
This paper introduces a method for domain adaptation, where each domain has noisy examples. Their method is based on a decision tree in which the data at each node are split into equal sizes while maximizing the
information gain.  They also proposed a way to reduce domain alignment. Their method is tested on several noisy domain adaptation settings and performs better than other baseline methods.
Pros:
Their idea to utilize a decision tree for domain adaptation sounds novel.
Experiments indicate the effectiveness of their method.
Cons:
This paper is not well-written and has many unclear parts.
1, The presentation of the problem set is unclear throughout this paper. In the abstract, they mentioned that they tackle the situation where both source and target domains contain noisy examples. However, they did not define the exact problem setting in any section. I could not understand what kind of problem setting motivated their method, which makes it hard to understand their method.
2, How they actually optimized the model is also unclear. From Eq 1~4, it is hard to grasp how they trained the model.
3, In open-set domain adaptation, simply minimizing domain-distance can harm the performance. How does the method avoid this issue? It was also unclear.
4, Experimental setting seems to be wrong and unclear. In Openset1, they say that "The labels from 1 to 10 of both source and target domains are marked as the known class, and all data with label 11∼20 in the source domain and label 21∼31 in the
target domain are used as one unknown class". However, Saito et al. (2018) used 21-31 classes in the target domain as one unknown class. In addition, "According to Saito et al. (2018) the target data of the unknown class is not used in training, ", they used the 21-31 classes for training in an unsupervised way. How is this method used to detect unknown class? Is there any threshold value set for it?
5, The experimental setting is unclear. In 4.4, ", we use only 10% of training samples", does it mean 10 % training source examples or target examples? This setting is also unclear.
From the cons written above, this paper has too many unclear parts in the experiments and method section. I cannot say the result is reproducible given the content of the paper and the result is a reliable one. They need to present more carefully designed experiments.


We present GraphMix, a regularized training scheme for Graph Neural Network based semi-supervised object classification, leveraging the recent advances in the regularization of classical deep neural networks. Specifically, we propose a unified approach in which we train a fully-connected network jointly with the graph neural network via parameter sharing, interpolation-based regularization and self-predicted-targets. Our proposed method is architecture agnostic in the sense that it can be applied to any variant of graph neural networks which applies a parametric transformation to the features of the graph nodes. Despite its simplicity, with GraphMix we can consistently improve results and achieve or closely match state-of-the-art performance using even simpler architectures such as Graph Convolutional Networks, across three established graph benchmarks: the Cora, Citeseer and Pubmed citation network datasets, as well as three newly proposed datasets : Cora-Full, Co-author-CS and Co-author-Physics.

Summary:
This paper presents a data augmentation procedure for semi-supervised learning on graph structured data. In general it is not clear how augmented observations can be incorporated into an existing graph while still preserving the graph structure. Instead of trying to incorporate an augmented dataset into the graph, this paper uses a seperate fully-connected network (FCN) that shares weights with a graph neural net (GNN). The final method also incorporates various components from the literature, such as Mixup and Sharpening. While the literature appears to have been adequately cited, the position of GraphMix relative to previous work could be better dilineated. In my opinion, the main contribution of this paper is the suggestion of using weight-sharing to sidestep the issue of incorporating augmented datapoints into graph-structured data. I am not sufficiently versed in the literature to assess whether the idea is sufficiently novel to warrant acceptance. The idea is simple and appears to improve the performance of Graph Convolutional Nets (GCNs), so I am leaning towards acceptance.
Rating Justification:
Efficient data-augmentation procedures are an important area of research. The relative simplicity and generality of GraphMix is appealing. I would give the paper a higher score if the authors showed that GraphMix(Graph U-net) was an improvement over Graph U-net, or if it was made more clear that some substantial benefit is derived from using Mixup features.
Additional Comments:
1. Based on the ablation study it seems that Mixup actually plays a very minor role in the overall success of the procedure. I would be curious to see the t-SNE visualization of the GraphMix(GCN) \ Mixup features in order to determine how much of the cluster separation is due to Mixup specifically. I understand that previous work has suggested that Mixup features are superior for discrimination than normal features, but in this work specifically the evidence for this assertion is fairly weak.
2. A clearer distinction between GraphMix(GCN) and GCN (with predicted targets) would be very helpful, especially since GCN (with predicted targets) actually performs the best on the standard PubMed splits. Why were the results for GCN (with predicted targets) not included in Table 2?


This paper introduces a new method to discover mislabeled training samples and to mitigate their impact on the training process of deep networks. At the heart of our algorithm lies the Area Under the Loss (AUL) statistic, which can be easily computed for each sample in the training set. We show that the AUL reliably reveals differences in training dynamics between (clean) samples that benefit from generalization and (mislabeled) samples that need to be "memorized". We demonstrate that the estimated AUL score conditioned on clean vs. noisy is approximately Gaussian distributed and can be well estimated with a simple Gaussian Mixture Model (GMM). The resulting GMM provides us with mixing coefficients that reveal the percentage of mislabeled samples in a data set as well as probability estimates that each individual training sample is mislabeled. We show that these probability estimates can be used to down-weight suspicious training samples and successfully alleviate the damaging impact of label noise. We demonstrate on the CIFAR10/100 datasets that our proposed approach is significantly more accurate and consistent across model architectures than all prior work.

In this paper, the authors proposed methodologies to identify mislabeled training examples. Specifically, the authors proposed to examine the training loss of each example to identify examples with high training losses. To mitigate the randomness in the training loss, the authors proposed an AUL (area under the loss) metric, which is less noisy that directly examining the loss. The authors proposed to use Gaussian mixture to approximate the AUL distribution, and then estimate the probability of mislabeling. The authors proposed to downweight training examples with high likelihood of being mislabeled. The proposal is examined in numerical studies.
While I agree the method has a good potential to be significant, in its current form, I have several concerns that prevent me from recommending acceptance.
1. How can we be sure that examples with large training loss are mislabeled examples, rather than examples that are inherently difficult to classify? For example, if the dataset consists of 99% bright images and 1% dark images, then it could be possible that dark image training examples have a higher loss. This question becomes more relevant when the model does not have sufficient complexity, or does not have the correct form of complexity.
2. If we cannot distinguish mislabeled examples with examples that are hard to classify, then it seems to me that a simple downweighting based on AUL could potential lead to the opposite outcome. Intuitively, we should upweight those difficult to classify examples to improve generalizability.
3. It is somewhat surprising to me that the AUL distribution is a nicely looking bimodal shape. Could it be due to the fact that the wrong labels are randomly generated with equal probabilities from other classes? What it we have a different error distribution, e.g., the label has p probability of being correct, and (1-p) probability of being k, where k is a constant?
4. Given the above concerns, I feel the numerical studies are insufficient. The authors may consider 1) including more noise distributions, 2) presenting the AUL distributions to check whether they are truly bimodal, and 3) trying more datasets and more model architectures.


Understanding the implicit bias of optimization algorithms is important in order to improve generalization of neural networks. One approach to try to exploit such understanding would be to then make the bias explicit in the loss function. Conversely, an interesting approach to gain more insights into the implicit bias could be to study how different loss functions are being implicitly minimized when training the network. In this work, we concentrate our study on the inductive bias occurring when minimizing the cross-entropy loss with different batch sizes and learning rates. We investigate how three loss functions are being implicitly minimized during training. These three loss functions are the Hinge loss with different margins, the cross-entropy loss with different temperatures and a newly introduced Gcdf loss with different standard deviations. This Gcdf loss establishes a connection between a sharpness measure for the 0 − 1 loss and margin based loss functions. We find that a common behavior is emerging for all the loss functions considered.

This paper adds to a large body of research on implicit regularization:
in the context of deep networks, that, when SGD with a certain step
size or batch size is used, from among parameter vectors that fit
the data equally well, some are much more likely to be chosen than
others.  This paper attacks this question through the lens of
loss functions: when SGD is applied to the usual softmax loss
with a given learning rate, how does the choice of learning
rate effect how rapidly other loss functions are reduced?  They
pay special attention to a loss function that they call Gcdf,
which is motivated by "wide minima" considerations.  In particular,
for the Gcdf to be small, not only must training examples be
classified correctly, but randomly perturbing the weights in
the last layer should not change this correct prediction.
I am convinced that Figures 6 and 12 of this paper show
that optimization of the softmax with a larger step size
implicitly optimizes a loss function that rewards robustness
to a greater extent than when a small step size is used.
I find this interesting.
The authors do a nice job of summarizing a lot of related work.
The Gcdf loss is similar to the ramp loss used in [1]
(see Section 3.1) and elsewhere, including to analyze
generalization in deep learning.  It is also like the
potential function optimized by RobustBoost
(see (4) of [2]) -- the RobustBoost loss function does
not scale by the norm of x, but since it is an ensemble
method, the role of x is played by the predictions of
members of the ensemble, which have a fixed scale.
I assume that, when they evaluate the Gcdf loss for a deep
network, they normalize by the norm of the last hidden
layer.  If this is true, it only captures "wide minima"
in the sense of being robust with respect to perturbations
of the output layer.  (They seem to acknowledge this point
in their paper.)
The results in the paper are not described in enough
detail to be reproduced.  For example, I don't see
where they specify the architecture of the network
that they used in Section 4.
The experiments are limited and narrow in scope.
In Figures 2 and 3 I don't see that they have adequately controlled
for the effect of the learning rate on how fast the explicitly
minimized loss is reduced.  Part of the effect observed is simply
that, when a small learning rate is used, after a given number of
epochs, the weights are just not changed much, so that no loss is
reduced much.  In Figure 2, I find it strange that they did not plot
the values for T=1.
Figure 6 is the most interesting to me.  It seems to show that
training the same loss function with a larger learning rate
effectively optimizes the gcdf loss that rewards sacrificing
training error to achieve stronger robustness.
Small point: the first time I read Appendix A, I thought that the
results were not there.  It was only later that I saw the figures with
the MNIST results.  It would be helpful if the authors wrote
"The results are in Figures 8-12".
While, as I wrote above, I did find Figure 6 interesting, I feel that
the increment of this research over the large body of previous work
on this topic is not enough to justify publication in ICLR.

[1] Bartlett, Peter L., Dylan J. Foster, and Matus
J. Telgarsky. "Spectrally-normalized margin bounds for neural
networks." Advances in Neural Information Processing Systems. 2017.
[2] https://arxiv.org/pdf/0905.2138.pdf


Multi-agent systems exhibit complex behaviors that emanate from the interactions of multiple agents in a shared environment. In this work, we are interested in controlling one agent in a multi-agent system and successfully learn to interact with the other agents that have fixed policies. Modeling the behavior of other agents (opponents) is essential in understanding the interactions of the agents in the system. By taking advantage of recent advances in unsupervised learning, we propose modeling opponents using variational autoencoders. Additionally, many existing methods in the literature assume that the opponent models have access to opponent's observations and actions during both training and execution. To eliminate this assumption, we propose a modification that attempts to identify the underlying opponent model, using only local information of our agent, such as its observations, actions, and rewards. The experiments indicate that our opponent modeling methods achieve equal or greater episodic returns in reinforcement learning tasks against another modeling method.

The authors propose a variational autoencoding (VAE) framework for agent/opponent modeling in multi-agent games. Interestingly, as the authors show, it looks like it is possible to compute accurate embeddings of the opponent policies without having access to opponent observations and actions. The paper is well written, the methods are simple yet still interesting/informative, but there are a few questions that I find necessary to be addressed.
Methods:
1. I find the idea of learning to infer embeddings of the opponent policies from the agent's own local observations quite interesting. Intuitively, it makes sense -- since the opponent's policy effectively specifies the dynamics of the environment (from the agent's perspective), opponent's behavior must be reflected in the agent's observations. Comparing figures 1 and 2, the proposed encoder architecture also uses information about the reward (and episode termination). How critical is this information for opponent identification? Would it work without r_{t-1} and d_{t-1}?

2. Sec. 4.2: "We assume a number of K provided episode trajectories for each pretrained opponent" -- how exactly are these trajectories obtained? Similarly, how exactly are the opponents pretrained? (Self-play, hardcoded, or something else?)
3. As the authors mention, the triplet loss that discriminates between the opponents loosens the lower bound. Since the regularized objective is still a lower bound, I wonder if the triplet loss can be re-interpreted/expresses as a specific prior on the opponent model?
Experiments:
1. Sec. 5.1: to understand the effect of opponent modeling, it would be nice to see how baselines perform in this setup against a randomly picked opponent (otherwise, the curve in Fig. 3-c is not informative). I suggest the following baselines: tit-for-tat (hardcoded), a couple of classical learning algorithms for iterated games (e.g., policy hill-climbing, WoLF), an agent that learns using policy gradients but without opponent embeddings. Without any baselines, Sec. 5.1 seems like a sanity check which just shows that the implementation works unless I am missing something.
2. Sec. 5.3: (1) Why is mutual information between the approximate posterior q and the prior p makes sense as the policy embedding quality metric here? (2) Could you intuitively (or formally) justify the fact that the triplet loss degrades MI metric? Right now, this is stated as a fact but not justified. (3) It looks like Grover et al. (2018a) used deterministic trajectory encoders; how exactly is MI measured in that case?
3. If I understand correctly from Fig. 4, SMA2C (which uses local information) underperforms as compared to the methods that use opponent trajectories in 6/8 cases. To me, this somewhat confirms the point opposite to what the authors claim -- local observations, while containing some information about the opponent, are still inferior. Also, having baselines that do not use opponent embeddings on the charts of Fig.4 would help understand the contribution of opponent modeling.
----
I acknowledge reading the author's response, which addressed some of my questions/concerns to some extent. However, I believe that while estimating accurate embeddings of the opponent behavior from the agent's observations only is interesting, the approach has limitations, and I feel those are not studied in-depth enough (e.g., as a reader, I would like to understand if and when I should use the proposed approach and expect it to work). My assessment of the paper stays the same.


Sign gradient descent has become popular in machine learning due to its favorable communication cost in distributed optimization and its good performance in neural network training. However, we currently do not have a good understanding of which geometrical properties of the objective function determine the relative speed of sign gradient descent compared to standard gradient descent. In this work, we frame sign gradient descent as steepest descent with respect to the maximum norm (L ∞ -norm). We review the steepest descent framework and the related concept of smoothness with respect to arbitrary norms. By studying the smoothness constant resulting from the L ∞ -geometry, we isolate properties of the objective which favor sign gradient descent relative to gradient descent. In short, we find two requirements on its Hessian: (i) some degree of "diagonal dominance" and (ii) the maximal eigenvalue being much larger than the average eigenvalue. We also clarify the meaning of a certain separable smoothness assumption used in previous analyses of sign gradient descent. Experiments verify the developed theory.

This paper analyzes the performance of sign gradient descent as a function of the “geometry” of the objective. They contrast the performance of sign GD to gradient descent. Both algorithms are steepest descent methods with respect to different norms. Overall, I do not think the insights claimed in the paper are especially novel. It is already known that the choice of a norm can have a significant impact on the speed of steepest descent but practically, the performance depends on quantities that are unknown such as the norm of the gradient over the specific trajectory of the algorithm. I do not find the arguments in section 4 especially convincing as they seem rather high-level to me. Instead I would find it more valuable if one could make a statement about the performance of signGD as a function of known quantities. One could perhaps start by analyzing a specific class of functions (quadratics, Polyak-Lojasiewicz functions, …).
Prior work
It seems to me that most results discussed in the paper are already known in the optimization community. The book by Boyd & Vandenberghe (cited by the authors) has an entire section on the “Choice of norm for steepest descent” where they indeed explain that the choice of the norm used for steepest descent can potentially have a dramatic effect on the convergence rate. Can the authors explain how they see their contribution compared to prior work? I could concede that the result of Proposition 3 is somewhat novel, although I think the authors should discuss prior work on axis-alignment, which dates back to
Becker, Sue, and Yann Le Cun. "Improving the convergence of back-propagation learning with second order methods." Proceedings of the 1988 connectionist models summer school. 1988.
Section 4
The discussion in section 4 is high-level and I don’t see any particular insight one gains over what’s already known (see again book by Boyd & Vandenberghe). As explained by the authors, the comparison between signGD and GD will depend on the trajectory being followed.
Stochastic setting
Can the authors comment on generalizing their results to a stochastic setting? Since the gradients have a larger norm when using the infinity norm, I would expect that the variance is also larger.
Experimental results
The empirical results on the neural networks do not seem especially convincing, and the authors do seem to acknowledge this. What aspect of the analysis or the experimental setup do you expect to be responsible for this? Have you considered optimizing smaller models first? (something in between the quadratic problem in section 5.1 and the neural net in section 5.2. Some sort of ablation study where one strips away various components (batch-norm, residual connections, ….) of a neural network would also be valuable.
Generalization
One aspect that might be worth investigating is the generalization ability of steepest descent for different norms. There is already some prior work for adaptive methods that could perhaps be generalized, see e.g.
Wilson, Ashia C., et al. "The marginal value of adaptive gradient methods in machine learning." Advances in Neural Information Processing Systems. 2017.


We present a causal view on the robustness of neural networks against input manipulations, which applies not only to traditional classification tasks but also to general measurement data. Based on this view, we design a deep causal manipulation augmented model (deep CAMA) which explicitly models the manipulations of data as a cause to the observed effect variables. We further develop data augmentation and test-time fine-tuning methods to improve deep CAMA's robustness. When compared with discriminative deep neural networks, our proposed model shows superior robustness against unseen manipulations. As a by-product, our model achieves disentangled representation which separates the representation of manipulations from those of other latent causes.

Summary:
(1) The paper makes vague and unsupported claims about causality.
   The notion of 'causal', as well as do() notation, is really hollowed out here. It is of limited significance. In Equation (3) do() disappears from one line to the next, illustrating how little it does.
(2) Lack of novelty: The paper references "Learning disentangled representations with Semi-supervised deep generative models", but fails to mention that their model is is nearly identical.
(3) The quality is on the low side; confusing mistakes in figures, in some
 sections poor proof-reading.
Details:
(1) Exaggerated claims.
"We argue that the incapability for causal reasoning is the reason of DNN's vulnerability".  (This is stated twice in different words).
  Unfortunately, the term "causal reasoning" is used without making it precise.  The paper has a very weak relation to causality; one could replace "causal" with "translation invariant", and very little would change.  Their experiments try to learn translation invariance (appendix shows whitening invariance).
The paper makes gratuitous use of do() notation; for example, regular MNIST is reinterpreted as not being observational, but as arising from an intervention, namely do(translation=0).  But do() does not amount to much here. It reduces to mere conditioning here, as the do is always applied to a root node without parents.  This is clear in equation in 3, where do's are correctly just deleted from one line to the next; do(), and 'causal' are used mostly as a rhetorical device.
(2) The model in the paper is pretty much the same as "Learning disentangled representations with Semi-supervised deep generative models", figure 2 here corresponds to figure 4 there.  Both papers use partially supervised (or unsupervised) VAEs. There are some differences, e.g. the prior work considers a more general case and uses importance sampling.  This paper refers but fails to say how similar they are.  There should be a comparison between the difference inference methods used.
(3) The paper is sloppy in places.
Figure 6: the dotted arrow between nodes A and Z should not be there.
It does not follow from the graphical model in Figure 5.
In Figure 12a),  the text contradicts the figure. The brown curve with fine tuning has lower accuracy than the green one without fine tuning, but the text says the opposite.
Figure 11: there is a curve mislabelled as "Dis"
Results are mixed.  While results on vertical translation invariance and adversarial robustness seem good, the method fails to achieve horizontal translation invariance (Figure 7 b) and c)) - disappointing. But some of the translation invariance seems to be due to just using a generative model (the VAE), as it occurs without any fine-tuning.