Localist coding schemes are more easily interpretable than the distributed schemes but generally believed to be biologically implausible. Recent results have found highly selective units and object detectors in NNs that are indicative of local codes (LCs). Here we undertake a constructionist study on feed-forward NNs and find LCs emerging in response to invariant features, and this finding is robust until the invariant feature is perturbed by 40%. Decreasing the number of input data, increasing the relative weight of the invariant features and large values of dropout all increase the number of LCs. Longer training times increase the number of LCs and the turning point of the LC-epoch curve correlates well with the point at which NNs reach 90-100% on both test and training accuracy. Pseudo-deep networks (2 hidden layers) which have many LCs lose them when common aspects of deep-NN research are applied (large training data, ReLU activations, early stopping on training accuracy and softmax), suggesting that LCs may not be found in deep-NNs. Switching to more biologically feasible constraints (sigmoidal activation functions, longer training times, dropout, activation noise) increases the number of LCs. If LCs are not found in the feed-forward classification layers of modern deep-CNNs these data suggest this could either be caused by a lack of (moderately) invariant features being passed to the fully connected layers or due to the choice of training conditions and architecture. Should the interpretability and resilience to noise of LCs be required, this work suggests how to tune a NN so they emerge.

Paper Overview:
This paper aims to study when hidden units provide local codes by analyzing the hidden units of trained fully connected classification networks under various architectures and regularizers.  The main text primarily studies networks trained on a dataset where binary inputs are structured to represent 10 classes with each input containing a subset of elements indicative of the class label.  The work also studies fully connected networks trained on the MNIST dataset (with the addition of some pixels indicating each class label).  After enumerating the number of local codes observed under these different settings, the authors conclude the following: (1) "common" properties of deep neural networks & modern datasets seem to decrease the number of local codes (2) specific architectural choices, regularization choices & dataset choices seem to increase the number local codes (i.e. increasing dropout, decreasing dataset size, using sigmoidal activations etc.).   The work then state that these insights may suggest how to train networks to have local codes emerge.
Review:
I particularly liked the simple dataset the authors construct in determining whether local codes emerge in hidden units, especially since deep networks and dataset used in practice are overly complex to gain insight for this behavior.  However, I find the overall message to be a bit confusing, especially in regard to using the analysis to construct networks with emergent local codes.  In particular, I feel that the authors could strengthen this work greatly by using their findings to train a deeper neural network for which local codes do emerge on a more realistic dataset.  Furthermore, this work would be significantly more impactful if a network with more local codes does generalize better, but that is unclear as of now (especially since local codes seem to not emerge in practical settings even though these networks are state of the art).
Criticisms/Questions:
(1) Main:  I'm somewhat confused about the main takeaway from this work in terms of understanding when local codes actually emerge in deep neural networks.  The authors seem to have a number of very specific conditions that are both architecture and dataset dependent, and overall I feel the message would be much stronger if the authors were able to rigorously study perhaps just a few of these conditions across many more settings.  For example, even just studying the impact of activation and providing some conditions/theory or a clearer understanding of which nonlinearities lead to more local codes would be insightful.  The current work seems to be more broad instead of tackling one of these properties in depth.
(2) I am a bit confused about the thresholds used by the authors in determining whether a hidden unit provides a local code or not. Do you just determine if there is some threshold given by the unit that separates out all points of one class from the rest?
(3) After several experiments, there are some heavy conjectures trying to rationalize the result of the experiment.  As an example, the authors provide statements like "ReLU is a more powerful activation function than sigmoid."  However, this statement in particular is not exactly correct, since given enough width, networks with either activation function should be able to interpolate the training data.  Another example of this is at the bottom of page 7, when the authors provide 5 possible explanations as to why local codes don't emerge in modern training settings.  It is unclear which of these explanations are true, but it would be great if the authors could actually provide a cleaner rationalization.
Minor criticisms:
(1) I've seen a number of different conventions for how to refer to the depths of networks, and I believe what you refer to as 3 layer networks would conventionally be referred to as 2-layer networks for theory audiences (as there are 2 weight matrices involved) or 1-hidden layer networks for empirical audiences.  I think adding a figure in the appendix for your architecture would clear up any confusion immediately.
(2) Some of the formatting is a bit awry: there are references to figures that appear as ?? (see page 8 paragraph 3).
(3) It would be nice to provide a consistent legend in some of the figures.  For example, Figure 4b has no indication for which settings the colors represent.
(4) As there seem to be a lot of experiments numbered 1-12, I think it would be much more readable to have different subsections on the different settings and outline the experiments in the subsection more clearly.  Referring back to these numbers on page 3 & 4 constantly makes it less readable.
(5) I quite liked Figure 8 in the Appendix.  I feel that this would have been a great figure to put towards the front of the paper to provide an example of local codes emerging.


Graph Neural Nets (GNNs) have received increasing attentions, partially due to their superior performance in many node and graph classification tasks. However, there is a lack of understanding on what they are learning and how sophisticated the learned graph functions are. In this work, we propose a dissection of GNNs on graph classification into two parts: 1) the graph filtering, where graph-based neighbor aggregations are performed, and 2) the set function, where a set of hidden node features are composed for prediction. To study the importance of both parts, we propose to linearize them separately. We first linearize the graph filtering function, resulting Graph Feature Network (GFN), which is a simple lightweight neural net defined on a set of graph augmented features. Further linearization of GFN's set function results in Graph Linear Network (GLN), which is a linear function. Empirically we perform evaluations on common graph classification benchmarks. To our surprise, we find that, despite the simplification, GFN could match or exceed the best accuracies produced by recently proposed GNNs (with a fraction of computation cost), while GLN underperforms significantly. Our results demonstrate the importance of non-linear set function, and suggest that linear graph filtering with non-linear set function is an efficient and powerful scheme for modeling existing graph classification benchmarks.

This paper presents a dissection analysis of graph neural networks by decomposing GNNs into two parts: a graph filtering function and a set function. Although this decomposition may not be unique in general, as pointed out in the paper, these two parts can help analyze the impact of each part in the GNN model. Two simplified versions of GNN is then proposed by linearizing the graph filtering function and the set function, denoted as GFN and GLN, respectively. Experimental results on benchmarks datasets for graph classification show that GFN can achieve comparable or even better performance compared to recently proposed GNNs with higher computational efficiency. This demonstrates that the current GNN models may be unnecessarily complicated and overkill on graph classification. These empirical results are pretty interesting to the research community, and can encourage other researchers to reflect on existing fancy GNN models whether it's worth having more complex and more computationally expensive models to achieve similar or even inferior performance. Overall, this paper is well-written and the contribution is clear. I would like to recommend a weak accept for this paper. If the suggestions below can be addressed in author response, I would be willing to increase the score.
Suggestions for improvement:
1) Considering the experimental results in this paper, it is possible that the existing graph classification tasks are not that difficult so that the simplified GNN variant can also achieve comparable or even better performance (easier to learn). This can be conjectured from the consistently better training performance but comparable testing performance of original GNN. Another possibility is that even the original GNN has larger model capacity, it is not able to capture more useful information from the graph structure, even on tasks that are more challenging than graph classification. However, this paper lacks such in-depth discussions;
2) Besides the graph classification task, it would be better to explore the performance of the simplified GNN on other graph learning tasks, such as node classification, and various downstream tasks using graph neural networks. This can help demystify the question raised in the previous point; 3) The matrix \tilde{A} in Equation 5 is not well explained (described as "similar to that in Kipf and Welling (2016)"). It would be more clear to directly point out that it is the adjacency matrix, as described later in the paper.


Over-parameterization is ubiquitous nowadays in training neural networks to benefit both optimization in seeking global optima and generalization in reducing prediction error. However, compressive networks are desired in many real world applications and direct training of small networks may be trapped in local optima. In this paper, instead of pruning or distilling over-parameterized models to compressive ones, we propose a new approach based on differential inclusions of inverse scale spaces, that generates a family of models from simple to complex ones by coupling gradient descent and mirror descent to explore model structural sparsity. It has a simple discretization, called the Split Linearized Bregman Iteration (SplitLBI), whose global convergence analysis in deep learning is established that from any initializations, algorithmic iterations converge to a critical point of empirical risks. Experimental evidence shows that SplitLBI may achieve comparable and even better performance than other training algorithms on ResNet-18 in large scale training on ImageNet-2012 dataset etc., while with early stopping it unveils effective subnet architecture with comparable test accuracy to dense models after retraining instead of pruning well-trained ones.

Summary
=======
This paper aims to train sparse neural networks efficiently, by jointly optimizing the weights and sparsity structure of the network. It applies the Split Linear Bregman Iteration (Split LBI) method from [1] in a large-scale setting, to train deep neural networks.
The approach works by considering optimization in a joint space (W, \Gamma) consisting of the network weights W and a new set of parameters \Gamma that model the structural sparsity of the network.
The problem of learning sparse networks efficiently is important for modern applications that run on embedded devices, as well as for fast training on specialized hardware. I think the approach is interesting as a potential alternative to more expensive methods for finding sparse networks, such as NAS and the successive pruning & re-training approach of [2].
Overall, the paper pursues a promising direction to induce network sparsity, and presents some interesting results. However, there are several issues with the experiments and the structure/presentation of the paper that should be addressed.
Pros
====
* As far as I am aware, this is the first application of Split LBI to train deep neural networks.
* It shows that joint optimization of the weights and sparsity structure performs on par with baselines that only optimize weights, on MNIST, CIFAR-10, and ImageNet.
* It provides a global convergence analysis that shows that the weights optimized with Split LBI converge to a critical point of the training loss, regardless of initialization.
* It provides an ablation study for the two hyperparameters \kappa and \nu of Split LBI.
* I think the most interesting parts of the paper are those that examine the structural sparsity learned by Split LBI (Sections 4.3 and 4.4). In particular, the fact that Split LBI was able to match or outperform the test accuracy of several baselines (network slimming, soft filter pruning, and the method in Rethinking the Lottery Ticket Hypothesis) in a single training run (without re-training) is a nice result.
Issues
======
* The Split LBI method is presented as a novel contribution (in the abstract: "we propose a new approach based on differential inclusions of inverse scale spaces ..."), but it was already described in detail in a paper that they cite ([1]) published at NeurIPS 2016. The only theoretical contribution of this paper is section 3: Global convergence of Split LBI.
* The claim that "Split LBI demonstrates SOTA performance in large scale training on ImageNet" (from the abstract) is not correct, and needs to be qualified. This paper reports 70.55/89.56% (top 1 / top 5) accuracy; as far as I am aware, the current SOTA on ImageNet is [3], which achieves 86.4/98.0% (top 1 / top 5) accuracy. I think it would be better for this paper to argue that it achieves comparable performance to baselines with a particular architecture and training regime.
* The structure and presentation of the paper could be improved in several ways, outlined as follows:
    - Most of the Methodology section discusses the Split LBI method from prior work. I would encourage the authors to split the Methodology section into a separate Background section for Split LBI, followed by a new section specifically about applying Split-LBI to convolutional and fully-connected layers in neural networks.
    - The writing is missing some details and explanations that would be very helpful for readers. For example, it should clearly state that the dimension of \Gamma is the same as the dimension of the weights W.
    - It would also be good to expand the explanation about why SplitISS avoids the parameter correlation problem, and what it means for \Gamma to have an orthogonal design?
    - The figures are too small to be readable without a lot of zooming.
    - The paper ends abruptly, with no conclusion.
    - The appendix contains some useful material, but much of it is not referenced from the main paper. I think some parts of the appendix could be moved to the main paper, for example the comparison of computational and memory costs between Split LBI and SGD.
* I am not sure what the purpose of the comparisons between optimizers in Table 1 is. The motivation given in the abstract and introduction is to learn sparse networks online during optimization; it does not propose Split LBI as a new optimizer to compete with Adam. Couldn't one use the Adam update rule to optimize the weights in Split-LBI? I think it makes sense to compare Split LBI to standard training setups that do not enforce any sparsity, as well as to setups that use L1 and L2 regularization, but I do not think that Table 1 is set up correctly for this. Different optimizers are paired with different regularizers, and crucially the choice of hyperparameters is not discussed---how did you choose the coefficient of L1 regularization to be 1e-3? Additionally, many rows in Table 1 are missing data (e.g., the variants of Adam are only run for CIFAR-10).
* Regarding experiments, it is not clear which experiments the authors actually performed, for which they took the results from previously published papers, and sometimes where the results come from at all.
    - In Table 1, there is an asterix next to SGD-Mom-Wd that the authors say indicates "results from the official pytorch website." That would imply that the rest of the experiments were done by the authors (and the authors say in the caption "we use the official pytorch codes to run the competitors"). However, Table 2 (found in the Appendix, page 20), contains identical numbers and a sign # that, according to the authors, indicates results of their own experiments. That would mean that of all the SGD and Adam experiments shown in the table, the authors only performed SGD-naive and Adam-naive. Where do the other numbers come from? What does "official pytorch code to run the competitors" mean? Where is that code from?
    - Figure 4 contains baselines and SplitLBI results. Where do the numbers for the baselines come from? The caption mentions another paper, [5], but I did not find the source of the numbers in that paper. The caption seems to point to Table 9a in [5], but that table does not deal with Network Slimming, Soft-Filter Pruning, Scratch B, or Scratch-E. Additionally, Table 9a of [5] only contains results for VGG-16 and ResNet-50. Where do the baselines for ResNet-56 (Figure 4b) come from?
* How is the proximal objective in Eq. 5 optimized? That is, how do you compute the argmin?
* Figure 2 shows results for SLBI-1 and SLBI-10, but no discussion of what SLBI-1 and SLBI-10 mean. Also regarding Fig.2, the authors claim that "Filters learned by ImageNet prefer non-semantic texture rather than shape and color." How did the authors come to this conclusion? I looked carefully at the filter visualizations, and I cannot see a clear difference between the filters learned by Split LBI and SGD.
* The computation time comparison in Table 11 (Appendix E) is a bit strange, because it shows that Adam takes 2x as long as SGD, which does not align with my experience; in practice, the wall-clock time is nearly identical between Adam and SGD. It would be good to provide more details about how the time was measured. Also, does the memory comparison measure only the memory used for model parameters (W and \Gamma), or also activation memory? Shouldn't Split LBI use 2x the memory of SGD (if measuring only the weights)?
* In Figure 1, it looks like the initial magnitude of the filters is larger for SGD compared to Split LBI. Are the weights initialized in the same way? Also, why is the setup of the MNIST experiment in Fig.1 different from the setup in Table 1 (e.g., learning rate decay every 40 epochs vs every 30 epochs)? In addition, it looks like the first learning rate decay causes the filter weight magnitudes to flatten out and stay constant.
* What is the learning rate schedule used for the runs in Figure 3? It looks like the lr decays at epoch 80 and 120, but this is only mentioned in table captions in the appendix. This should be stated in the main paper. Also, why is this a different training setup from that used for Table 1?
I also noted that the authors do not intend to make the code public upon publication of the paper. On page 6, they state that "source codes will be released upon requests." At present, the preferred path is to make the code public upon publication of the paper.
Minor points
============
* In the caption of Figure 3, it says "The results are repeated for 5 times. Shaded area indicates the variance; and in each round, we keep the exactly same initialization for each model." What is different between the 5 runs if the initialization is the same?
* There are too many different colors used in Figures 1 and 2. Since the purple, green, and black boxes are important to see for figures 1 and 2, it is confusing to have to deal with additional blue, pink, and yellow boxes around every three.
[1] Huang et al., Split LBI: An iterative regularization path with structural sparsity. NeurIPS 2016.
[2] Frankle & Carbin, The Lottery Ticket Hypothesis: Finding sparse, trainable neural networks. ICLR 2019.
[3] Touvron et al., Fixing the train-test resolution discrepancy. https://arxiv.org/abs/1906.06423.
[4] He et al., Deep residual learning for image recognition. https://arxiv.org/abs/1512.03385.
[5] Liu et al., Rethinking the value of network pruning. ICLR 2019.
Post-rebuttal Update
====================
I thank the authors for their rebuttal, and for clarifying some details in the paper.
* I think the experiments on sparsity are interesting. More efficient ways to find good sparse networks are certainly of interest to the community.
* I appreciate that the authors released the source code.
* In summary, this paper applies Split-LBI to neural network training, and provides a global convergence result as one of the main contributions. Operationally, compared to the original Split LBI approach, it changes the loss function from squared error to cross entropy, and uses mini-batches for training, which are fairly straightforward.
* One important issue with the paper is that it blurs the distinction between prior work and the new contribution. For example, the subsection on Split Linearized Bregman Iteration in the "Methodology" section does not contain anything new compared to [1], and this is not clear enough to the reader. Also, not enough credit is given to [2] for the "Differential of Inclusion of Inverse Scale Space" subsection. I maintain that there needs to be a separate "Background" section for this, and that it should be made absolutely clear what is new in the "Methodology" section. It feels like this distinction is obfuscated in the writing.
* Given that the authors propose Split-LBI as a new optimizer that can be compared to others (e.g., Adam), one issue is that there doesn't seem to be any search done over the hyperparameters of each optimizer, including the learning rate and amount of weight decay. For example, Adam is only run with learning rate 1e-3 and SGD is run with 0.1; in addition, the weight decay (where used) is only set to 1e-4. Thus, it is not clear how meaningful these comparisons are. Also, in Table 1, the CIFAR-10 test accuracies are fairly low at ~90%, while modern models such as Wide ResNets can achieve ~95%.
* The newly-written conclusion is still incorrect, stating again that Split LBI achieves SOTA performance on ImageNet. Also, if  "with better interpretability than SGD" refers to the qualitative comparison of the learned filters, I think this conclusion is a bit too strong, because I don't think the difference is visible enough to aid interpretability.
* Minor point: On further inspection, the legend in the left-side plots in Figure 2 does not match the labels of the visualizations on the right. There is no yellow training curve in Figure 2, despite the assertion in the rebuttal.
[1] Huang et al., Split LBI: An iterative regularization path with structural sparsity. NeurIPS 2016.
[2] Osher et al., "Sparse recovery via differential inclusions." Applied and Computational Harmonic Analysis, 2016.
I maintain my score of weak reject, but am not totally opposed to it being accepted, because it provides a way to find sparse networks more efficiently.


Hierarchical abstractions are a methodology for solving large-scale graph problems in various disciplines. Coarsening is one such approach: it generates a pyramid of graphs whereby the one in the next level is a structural summary of the prior one. With a long history in scientific computing, many coarsening strategies were developed based on mathematically driven heuristics. Recently, resurgent interests exist in deep learning to design hierarchical methods learnable through differentiable parameterization. These approaches are paired with downstream tasks for supervised learning. In practice, however, supervised signals (e.g., labels) are scarce and are often laborious and expensive to obtain. In this work, we propose an unsupervised approach, coined OTCOARSENING, with the use of optimal transport. Both the coarsening matrix and the transport cost matrix are parameterized, so that an optimal coarsening strategy can be learned and tailored for a given set of graphs. We demonstrate that the proposed approach produces meaningful coarse graphs and yields competitive performance compared with supervised methods for graph classification and regression.

This paper proposes an unsupervised hierarchical approach for learning graph representations. The proposed architecture is constructed by unrolling k-steps of a parametrized algebraic multigrid approach for minimizing the Wasserstein metric between the graph and its representation. The node distance (transport cost) used in the Wasserstein metric is also learned as an L2 distance between the embeddings of some graph embedding function. The approach is compared against 6 other state of the art approaches on 5 graph classification tasks, showing significant improvements 4 of them.
The paper is reasonably well written, however, I think some of the explanations can be tightened further. Especially a lot on the background of AMG is not really that relevant, since the authors are not transferring technical results from AMG. Also, it seems like a better flow for presenting this argument might be to switch the order of sections 3.2.1 and 3.1.2. It looks like the main point is  that this architecture is trying to emulate iterative coarsened residual optimization of the Wasserstein metric between a graph and its representation. How the coarsening matrix is derived is more of a technical point (it looks like the results would be much more sensitive to a switch of metric than to a switch of parametrization for S).
The empirical results are quite intriguing. There are, however, natural and important questions left unanswered. First and foremost, how does the amount of downsampling (compression) compare between methods. How many parameters do different methods require? It would also be good to see what the baseline performance would have been without any input compression as to understand how close these approaches are to the upper bound.
Finally, I think the main issue of this paper, is left unresolved, namely, what is the point of not having supervision from the downstream task. As a user of graph representations trying to solve some problem, the only thing I would want from my representation is to capture some notion of sufficient statistics that are small enough to be efficient and allow me to solve my problem. I would not necessarily care about how well the learned representation resembles the original graph unless I believed that my downstream task was hard to evaluate  and that it was very smooth in the Wasserstein metric. I read the paper multiple times, trying to find any discussion on this, but it seems that the fact that an unsupervised representation is a good thing is taken for granted. A point could at least be made using the same representation for different tasks experimentally. Or, perhaps, literally doing an AMG-type unpacking of the downstream task itself as a comparison. This would shed light on the question of whether the iterated residuals or the choice of distance is what's driving the observed results.


A channel corresponds to a viewpoint or transformation of an underlying meaning. A pair of parallel sentences in English and French express the same underlying meaning but through two separate channels corresponding to their languages. In this work, we present Multichannel Generative Language Models (MGLM), which models the joint distribution over multiple channels, and all its decompositions using a single neural network. MGLM can be trained by feeding it k way parallel-data, bilingual data, or monolingual data across pre-determined channels. MGLM is capable of both conditional generation and unconditional sampling. For conditional generation, the model is given a fully observed channel, and generates the k − 1 channels in parallel. In the case of machine translation, this is akin to giving it one source, and the model generates k − 1 targets. MGLM can also do partial conditional sampling, where the channels are seeded with prespecified words, and the model is asked to infill the rest. Finally, we can sample from MGLM unconditionally over all k channels. Our experiments on the Multi30K dataset containing English, French, Czech, and German languages suggest that the multitask training with the joint objective leads to improvements in bilingual translations. We provide a quantitative analysis of the quality-diversity trade-offs for different variants of the multichannel model for conditional generation, and a measurement of self-consistency during unconditional generation. We provide qualitative examples for parallel greedy decoding across languages and sampling from the joint distribution of the 4 languages.

[Paper summary]
This work is an extension of KERMIT (Chan et al., 2019) to multiple languages and the proposed model is called “multichannel generative language models”. KERMIT is an extension of “Insertion Transformer” (Stern et. al, 2019), a non-autoregressive model that can jointly determine which word and which place the translated words should be inserted. KERMIT shares the encoder and decoder of insertion Transformer, and the source sentence and target sentence are concatenated to train a generative model (also, various loss functions are included). In this work, parallel sentences from more than two languages are concatenated together and fed into KERMIT. Each language is associated with a language embedding. This work demonstrates that a joint distribution p(x1, . . . , xk) over k channels/languages can be properly modeled through a single model. The authors carry out experiments on multi30k dataset.
[Pros] Some discoveries of this work are interesting, including: (1) It is possible to use a single model to translate a sentence into different languages in a non-autoregressive way. (2) The unconditional multilingual generation in Section 4.5 is interesting, especially, the generation order is determined by the model rather than left-to-right.
[Questions]
1.	The authors work on multi30k dataset, which is not a typical dataset for machine translation.
(A)	The dataset and the corresponding information is at https://github.com/multi30k/dataset. The number of words in a sentence is smaller than 15, which is too short for a machine translation. Also, the pattern of sentences is relatively simple.
(B)	For real world application, I am not sure whether it is possible to collect a large amount of k-parallel data where $k>2$. Therefore, the application scenario is limited. What if we have a large amount of bilingual data instead of k-parallel data? How should we leverage the large amount of monolingual data?
2.	For novelty, this is an extension of KERMIT to a multilingual version, which limits the novelty of this wok.
3.	The best results on En->De in Table 1 are inconsistent. On tst16, bilingual en<->de is the best; on tst17, en<->{rest} is the best; on mscoco, any<->rest is the best. In Table 2, seems using bilingual data only is the best choice. This makes me confuse about how to use your proposed method. However,


Despite alarm over the reliance of machine learning systems on so-called spurious patterns in training data, the term lacks coherent meaning in standard statistical frameworks. However, the language of causality offers clarity: spurious associations are those due to a common cause (confounding) vs direct or indirect effects. In this paper, we focus on NLP, introducing methods and resources for training models insensitive to spurious patterns. Given documents and their initial labels, we task humans with revising each document to accord with a counterfactual target label, asking that the revised documents be internally coherent while avoiding any gratuitous changes. Interestingly, on sentiment analysis and natural language inference tasks, classifiers trained on original data fail on their counterfactually-revised counterparts and vice versa. Classifiers trained on combined datasets perform remarkably well, just shy of those specialized to either domain. While classifiers trained on either original or manipulated data alone are sensitive to spurious features (e.g., mentions of genre), models trained on the combined data are insensitive to this signal. We will publicly release both datasets.

This paper addresses the problem of building models for NLP tasks that are robust against spurious correlations in the data by introducing a human-in-the-loop method: annotators are asked to modify data-points minimally in order to change the label.  They refer to this process as counterfactual augmentation.  The authors apply this method to the IMDB sentiment dataset and to SNLI and show (among other things) that many models cannot generalize from the original dataset to the counterfactually-augmented one.
This contribution is timely and addresses a very important problem that needs to be addressed in order to build more robust NLP systems.
Because, however, of a few limitations, I recommend weak acceptance.
My main hesitation comes from a lack of clarity about the main lesson we have learned.  In particular, if the goal is to use this method to augment the data we use to train NLP systems in order to make them more robust, it seems that the time cost of the process will be prohibitive.  On the other hand, perhaps these methods could be used to identify the kind of spurious correlations that models tend to rely on, which could then be used in a more automated data augmentation process.  If that's the goal, however, a more detailed error analysis would need to be included.
A few small comments:
* There was some analysis of the augmented IMDB dataset, but none of the SNLI dataset.  I would love to see a more detailed investigation of what annotators usually did.  For instance, a reason that hypothesis-only models do well is that certain words are very predictive of certain labels (e.g. "not" and contradiction).  Do people leave the negations in when modifying such examples for entailment or neutrality, thus breaking the simple correspondence?  That's a very simple kind of question; more generally, I'd like to see more analysis of the new dataset.
* The BiLSTM they use is very small (embedding and hidden dimension 50).  Given that BERT is most robust against their manipulation, it would be good to see a more powerful recurrent model for comparison.  It would be easy to use ELMo here, if the main question is about Transformers vs recurrent models.
Some very minor / typographic comments:
* abstract: "with revise" should be "with revising"
* first paragraph page 2: some references to causality literature and definition of spuriousness as common cause
* page 2, "We show that..." I'd break this into two sentences to make it easier to parse.
* Table 3: I would make two columns for each model with accuracy on original versus revised.  With the current table, one has to compare cells in the top half of the table to those in the bottom half of the table, which is quite difficult to do.


Learning rich representation from data is an important task for deep generative models such as variational auto-encoder (VAE). However, by extracting high-level abstractions in the bottom-up inference process, the goal of preserving all factors of variations for top-down generation is compromised. Motivated by the concept of "starting small", we present a strategy to progressively learn independent hierarchical representations from high-to low-levels of abstractions. The model starts with learning the most abstract representation, and then progressively grow the network architecture to introduce new representations at different levels of abstraction. We quantitatively demonstrate the ability of the presented model to improve disentanglement in comparison to existing works on two benchmark data sets using three disentanglement metrics, including a new metric we proposed to complement the previously-presented metric of mutual information gap. We further present both qualitative and quantitative evidence on how the progression of learning improves disentangling of hierarchical representations. By drawing on the respective advantage of hierarchical representation learning and progressive learning, this is to our knowledge the first attempt to improve disentanglement by progressively growing the capacity of VAE to learn hierarchical representations.

This paper proposed a method for training Variational Ladder Autoencoder (VLAE) using a progressive learning strategy. In comparison to the generative model using a progressive learning strategy, the proposed method focuses not only on the image generation but also on extracting and disentangling hierarchical representation.
Overall, I think the purpose of this paper should be written clearly. It is not clear whether the purpose is learning the disentangled representation or the hierarchical representation. In my opinion, I think the focus of the proposed method lies in the hierarchical representation through progressive learning, but the experiments are involved more with disentanglement. Furthermore, I believe the authors need to explain the relationship between hierarchical representation and disentangled representation. In particular, it is not clear why learning hierarchical representation is helpful for disentangled representations.
The qualitative experiments are not convincing since the proposed model looks worse in both the reconstruction and hierarchical disentanglement for MNIST dataset than the base model VLAE, as shown in Figure 5 in [1]. Regarding the metric used in the experiments, the authors mention that the proposed disentanglement metric MIG-sup is what they first developed for one-to-one property, but it seems that it was already proposed in [2]. In addition, the proposed metric requires ground truth for the generative factors, so its usage is limited and not practical.
I think this work is similar to [3] in that both learn disentangled representations by progressively increasing the capacity of the model. I think the authors need to discuss about this work.
Ablation studies should be presented to verify the individual effects of the progressive learning method and implementation strategies on performance, respectively.
In Figures 2 and 3, the performance gap in the reconstruction error of the proposed method is greater than the base model when beta changes from 20 to 30. Therefore, it is necessary to show if it is robust against the hyperparameter beta.
There is no definition of v_k in Equation (12), so it is difficult to understand the proposed metric clearly.
In summary, I do not think the paper is ready for publication.
[1] Learning Hierarchical Features from Generative Models, Zhao et al., ICML 2017
[2] A Framework for the Quantitative Evaluation of Disentangled Representations, Eastwood et al., ICLR 2018
[3] Understanding disentangling in beta-VAE, Burgess et al., NIPS 2017 Workshop on Learning Disentangled Representations
-------------------------------------
After rebuttal:
Thanks for the revision of the paper and the additional experiments.
The authors' comments and further experiments address most of my concerns. In particular, new experiments show that pro-VLAE performs quantitatively and qualitatively better than VLAE. Also, Figure 10 and the result of the information flow experiment using MNIST show that the first layer learns the intended representations properly.
I appreciate the authors’ efforts put into the rebuttal, and the results of additional experiments are reasonably good. Therefore, I increase my final score to 6: Weak Accept.


Latent-variable models represent observed data by mapping a prior distribution over some latent space to an observed space. Often, the prior distribution is specified by the user to be very simple, effectively shifting the burden of a learning algorithm to the estimation of a highly non-linear likelihood function. This poses a problem for the calculation of a popular distance function-the geodesic between data points in the latent space-as this is often solved iteratively via numerical methods. These are less effective if the problem at hand is not well captured by first or secondorder approximations. In this work, we propose less complex likelihood functions by allowing complex distributions and explicitly penalising the curvature of the decoder. This results in geodesics which are approximated well by the Euclidean distance in latent space, decreasing the runtime by a factor of 1,000 with little loss in accuracy. Additionally, we apply our method to a state-of-the-art tracking algorithm using real world image data, showing that our unsupervised method performs similar to supervised learning methods.

Summary of paper:
The paper is concerned with the geometry of latent spaces in VAEs. In particular, it is argued that since geodesics (shortest paths) in the Riemannian interpretation of latent spaces are expensive to compute, then it might be beneficial to regularize the decoder (generator) to be flat, such that geodesics are straight lines. One such regularization is proposed.
Review:
I have several concerns with the paper:
1) Geodesics are never motivated:
The paper provides no motivation for why geodesics are interesting objects in the first place, so it is not clear to me what the authors are even trying to approximate.
2) Under the usual motivation, the work is flawed:
The usual motivation for geodesics is that they should follow the trend of the data (e.g. go through regions of high density). Since no other motivation is provided, I will assume this to be the motivation of the paper as well. The paper propose to use a flexible prior and then approximate geodesics by straight lines. Beyond the most simple linear models, then this cannot work. If the prior is flexible, then straight lines will hardly ever constitute paths through regions of high density. The core idea of the work, thus, seem to be in conflict with itself.
3) A substantial bias is ignored:
The paper consider the Riemannian metric associated with the *mean* decoder. Due to regularization, holes in the data manifold will be smoothly interpolated by the mean decoder, such that geodesics under the associated metric will systematically be attracted to holes in the data manifold. Hauberg discuss this issue in great length here:
  https://arxiv.org/abs/1806.04994
Here it is also demonstrated that geodesics under the mean decoder tend to be straight lines (which is also what the authors observe). Taking the stochasticity of the VAE decoder into account drastically change the behavior of geodesics to naturally follow the trend of the data.
4) Related work is mischaracterized:
Previous work on the geometry of latent spaces largely fall into two categories: those that treat the decoder as deterministic and those that treat it as being stochastic. In the cited papers Arvanitidis et al and Tosi et al consider stochastic decoders, while the other consider deterministic decoders. Given that geodesics have significantly different behavior in the two cases, it is odd that the difference is never discussed in the paper.
5) It is not clear to me what the experiments actually show:
-- I did not understand the sentence (page 5): "The model is more invariant if the condition number is smaller..." What does it mean to be "more invariant" ? And how is invariance (to what) related to the condition number of the metric?
-- Figure 3 show example geodesics, but only geodesics going between clusters (I have no idea how such geodesics should look). If I look at the yellow cluster of Fig3a, then it seems clear  to me that geodesics really should be circular arcs, yet this is being approximated with straight lines. Are the ground truth geodesics circular? At the end, it seems like the shown examples are the least informative ones, and that intra-cluster geodesics would carry much more meaning.
-- What am I supposed to learn from the "Smoothness" experiment (page 7) ? My only take-away is currently that the proposed regularization does what it is asked to do. It is not clear to me if what it aims to do is desirable? Does the experiment shed light on the desirability of the regularizer or is it more of a "unit test" that show that the regularizer is correctly implemented?
-- In the "Geodesic" experiment (page 7) I don't agree with the choice of baseline. If I understand correctly, the baseline approximate geodesics with shortest paths over the neighbor graph (akin to Isomap). However, there is no reason to believe that the resulting paths bare any resemblance to geodesics under the studied Riemannian metric. The above-mentioned paper by Hauberg provide significant evidence that these baseline geodesics are not at all related to the actual geodesics of the studied metric. The only sensible baseline I can think of is the expensive optimization-based geodesics.
== rebuttal ==
I have read the rebuttal and discussed with the authors, and I retain my original score.


Generating visualizations and interpretations from high-dimensional data is a 1 common problem in many fields. Two key approaches for tackling this problem approaches. We present a new deep architecture for probabilistic clustering, VarP-7 SOM, and its extension to time series data, VarTPSOM. We show that they achieve  way, a much higher degree of confidence in the findings of the exploration is attained (Keim, 2002).

This paper proposes VarPSOM, a method which utilizes variational autoencoders (VAEs) and clustering techniques based on self-organizing maps (SOMs) to learn clustering of image data (MNIST and Fashion MNIST in particular). An LSTM-based extension termed VarTPSOM is also evaluated on medical time series data. For the most part, the experimental results are promising, and the visualizations are particularly nice.
One of my main points of confusion is with the exposition of the method. To start, the objective presented in Eq. 3 is simply a sum of a variational lower bound and the PSOM clustering loss. Does this have a probabilistic interpretation, e.g., is it a lower bound for a particular generative model? If so, this would be useful to discuss prominently in the paper. If not, it is not clear to me what the authors are gaining from the variational framework. The paragraph at the bottom of page 4 that discusses the "advantages of a VAE over an AE" is not convincing to me. The authors claim that "points with a higher variance in the latent space could be identified as potential outliers and therefore treated as less precise and trustworthy". This isn't demonstrated in the experiments, and to the best of my knowledge, this has not been shown in prior work. If I am mistaken, a citation would be appreciated and should be included in the paper. Additionally, the claim that "the regularization term of the VAE prevents the network from scattering the embedded points discontinuously in the latent space" can also be accomplished with AEs with simple regularization, a standard technique for a wide range of AEs.
Similar comments can be made for the VarTPSOM objective in Eq. 6. Prior work in variational inference for time series, e.g., [1, 2] define a probabilistic time series generative model, from which variational inference naturally prescribes a learning objective. In my opinion, this stands in stark contrast to this work, which takes the VarPSOM objective and simply adds a time series loss on top. This is also a viable approach to building models, but why emphasize variational so much if the method is hardly motivated by anything variational?
I believe that the authors need more thorough experimental comparisons if they wish to demonstrate that their method actually benefits from the variational pieces. Most obviously, I do not believe that any of the comparisons represent the proposed method but with the VAE swapped out for some type of AE? It is my understanding that AE+SOM, SOM-VAE, and DESOM do not represent this exact ablation. VarIDEC performing better than IDEC is a data point in support of this hypothesis, however, this is a comparison of a prior method and not the proposed method.
The related work section mentions that SOM-VAE and DESOM are "likely limited by the absence of techniques used in state-of-the-art clustering methods". Is it possible to address this limitation of prior work? If so, how would this approach compare to the proposed method in terms of implementation and performance? I am not necessarily interested in an actual empirical evaluation, but including this in the related work section would likely be interesting for the reader.
The authors claim in the implementation details that "[s]ince the prior in the VAE enforces the latent embeddings to be compact, it also requires more dimensions to learn a meaningful latent space". Is there a citation for this? My understanding is that posterior collapse leads to VAEs not using additional dimensions even when they are provided, which seems to contradict this claim.
Table 2 seems to have very low NMI numbers across the board, am I reading this incorrectly? Are there prior SOTA numbers that can be included?
Finally, it seems that some of the ideas and motivation in the paper are related to learning discrete structures with variational approaches, e.g., [3, 4]. If the authors agree, it may be appropriate to include some discussion in related work.
[1] Johnson et al, "Composing graphical models with neural networks for structured representations and fast inference". NIPS 2016.
[2] Fraccaro et al, "Sequential neural models with stochastic layers". NIPS 2016.
[3] Tomczak and Welling, "VAE with a VampPrior". AISTATS 2018.
[4] Vikram et al, "The LORACs prior for VAEs: Letting the trees speak for the data". AISTATS 2019.
------
To elaborate on my "Experience Assessment" of "I have read many papers in this area": "this area" in my case refers to amortized variational inference and VAEs, not clustering techniques and SOM.


This paper studies the undesired phenomena of over-sensitivity of representations learned by deep networks to semantically-irrelevant changes in data. We identify a cause for this shortcoming in the classical Variational Auto-encoder (VAE) objective, the evidence lower bound (ELBO). We show that the ELBO fails to control the behaviour of the encoder out of the support of the empirical data distribution and this behaviour of the VAE can lead to extreme errors in the learned representation. This is a key hurdle in the effective use of representations for data-efficient learning and transfer. To address this problem, we propose to augment the data with specifications that enforce insensitivity of the representation with respect to families of transformations. To incorporate these specifications, we propose a regularization method that is based on a selection mechanism that creates a fictive data point by explicitly perturbing an observed true data point. For certain choices of parameters, our formulation naturally leads to the minimization of the entropy regularized Wasserstein distance between representations. We illustrate our approach on standard datasets and experimentally show that significant improvements in the downstream adversarial accuracy can be achieved by learning robust representations completely in an unsupervised manner, without a reference to a particular downstream task and without a costly supervised adversarial training procedure.

This is a very interesting paper, I believe, a solid contribution to Variational Autoencoders. The basic argument is that encoders in VAEs are highly susceptible to noise in input data, whereas decoders are not. This argument is supported with a full fledged section 2.2, reformulating ELBO objective of VAEs, and introducing a VAE with discrete latent variables and discrete observations, so as to easily understand why and where VAEs fail.
To make encoders robust to noise in inputs, it is proposed to generate new fictive data points in the neighborhood of original data points so as to ensure that the latent representations of a data point and its fictive version are similar in "some sense" as part of the proposed regularization term. The implementation of this idea is solid in the paper, relating it to theoretical concepts such as  "entropy regularized entropy transport problem", "Wasserstein distance", etc. The most important point is that, it is easy to extend an encoder of an existing VAE with the proposed algorithm, while letting a decoder be untouched as the latter is shown to be robust/smooth anyways (in sec 2.2). It is also discussed on how to generate fictive samples, including but not restricted to approaches like projected gradient descent based adversarial attacks.
Section 2.2 can be improved further, in terms of presentation. This is the most important section which can be of interest to the community to understand VAEs' limitations, a good contribution on its own. Though challenging, I encourage the authors to improve the exposition in this section as much as possible.
Introduction is written beautifully. Good job, done!
For instance, some explanation about variables, m_j, u_i, their distribution.
How do you relate the Eq. 1 with the standard ELBO. (some reference to derivation?)
Is it not possible to explain limitations of present VAEs without introducing the particular von Mise like parameterization (last equation of page 3). I am not suggesting that you should remove it. The connections between the two could be more explicit, though I understand that it is already mentioned in the paper, "parameterization emulates a high capacity network that can model any functional relationship between latent states and observations...".
In this context, I found the explanation after Eq. 2 to be intuitive in regards to inefficiency of encoders. If I understand correctly, to put it in even simpler terms, the encoding neural network is overfitting mapping from input data points to the latent representations, not performing any learning for the unseen data points at all; on the other hand, decoder explores the space of latent variables well because it is modeled as a Gaussian?
Some of the new equations should be numbered for easy reference.
On page 4, the flow is a bit abrupt. Right after Fig. 3, there are points 1 and 2 added without any note on what these two points (items in latex) are about.
I found point 1 very confusing in page 4. On the other hand, point 2 is beautifully written. Though, it could be made explicit in the latter on why encoders found in VAE are not smooth, referring to Fig 2, 3.
There are minor grammar mistakes making some of sentences incoherent or confusing, in the paper. Something to do with style of language. I think, overall, language can be improved. Though, technical flow of the paper is great, and introduction is written very well, pointing out very important bold insights about the literature on unsupervised representation learning. I would say, it is a very well written paper, which is an enjoyable read, despite some of the grammar mistakes which can be easily fixed by proof reading.
Experimental evaluation is sufficient.
Last but not the least, one could argue that we are going to the literature of kernel function based methods, or markov random fields, to improve the neural network models. This is a general trend we are observing. It is interesting to see new models such as the proposed one, getting the best from both worlds. It may be worthwhile to point  out something along these lines in the paper so that other works like this can be accomplished which are bold, and advance representation learning, digging mathematical concepts from diverse domains. If I am mistaken, please feel free to point out. It is not going to be change the review. I am inspired from this work.

One practical challenge is to generate fictive data points which are not very near to existing data points. I am not sure if GANs can achieve that, either. Having such points is critical to deal with more structured noise. Any comments on this?


Training certifiable neural networks enables one to obtain models with robustness guarantees against adversarial attacks. In this work, we use a linear approximation to bound model's output given an input adversarial budget. This allows us to bound the adversary-free region in the data neighborhood by a polyhedral envelope and yields finer-grained certified robustness than existing methods. We further exploit this certifier to introduce a framework called polyhedral envelope regularization (PER), which encourages larger polyhedral envelopes and thus improves the provable robustness of the models. We demonstrate the flexibility and effectiveness of our framework on standard benchmarks; it applies to networks with general activation functions and obtains comparable or better robustness guarantees than state-of-the-art methods, with very little cost in clean accuracy, i.e., without over-regularizing the model.

This paper proposes a certifiable NN training method, "polyhedral envelope
regularization" (PER) for defending against adversarial examples.  The defense
is based on the same linear relaxation based outer bounds of neural networks
(KW/CROWN) used in many previous works.  The paper makes a few new (but small)
technical contributions:
1. this paper uses a different loss function (7), which is essentially Hinge
loss on the lower bounds of distance to decision boundary. Previous works like
KW used cross-entropy loss on the lower bound of prediction margin instead,
which was based on minimax robust optimization theory. But I am not fully
convinced if the new loss function is better or not.
2. in (5), the authors solve the bounded input case more carefully than
previous works.  (5) is trivial to solve in the L infinity case and has been
used in previous works like (Wong & Kolter 2018, Gowal et al., 2018 and Zhang
et al., 2019); but solving it for other norms requires some efforts, and this
paper proposes a good solution for it (Algorithm 2);
3. In previous works like KW/CROWN, to find the largest certifiable radius, a
binary search is needed. The authors proposes a very small improvement to the
binary search process by setting the lower bound of search to the largest
epsilon that is certifiable using the current linear relaxations obtained from
a larger epsilon.
The authors does not improve any bounds proposed in KW/CROWN, and they reuse
the same bounds. I see the main contribution as the new hinge-like loss
function for training, and a more careful procedure to find the largest
certifiable radius in bounded input case.
Empirically, the improvement of the proposed algorithm is limited - based on
Table 1 it is hard to say if PER is better than KW or not. PER+at outperforms
KW sometimes, however it is not a completely fair comparison, as we can add a
PGD based adversarial training loss to KW as well, as done in DiffAI (Mirman et
al., https://github.com/eth-sri/diffai).
Questions:
1. In my personal experience I usually found Hinge loss not as effective as
cross-entropy loss in deep learning based tasks probably due to its
non-smoothness. The claim that (7) is better than cross-entropy loss is that it
does not overregularize the network. The authors should provide more evidence
to show if this argument holds, e.g., plotting the norm of weight matrices
during the training for the two losses to show that it can reduce
overregularization.
2. I think the metric ACB KW and ACB CRO (average certified radius of KW/CROWN)
in Table 1 and 2 are confusing and not fair. In KW and CROWN's evaluation,
giving an epsilon, if an example cannot be certified due to epsilon to large
(i.e., ||A|| \epsilon + b > 0), certifiable radius will be count as 0 (flat
line in Figure 1(a)). In this paper, the authors instead in this case use -b /
||A|| as the certifiable radius. This is merely a different way of evaluation,
and I don't see this as a contribution, as the "improvement" does not come from
a tighter bound.  In the same sense, I don't think Figure 1(a) and the
discussions on page 3 are appropriate characterization of KW/CROWN. PEC uses
exactly the same linear bounds as in KW/CROWN, and has the same certification
power.
3. For L2 based perturbations, in Table 1, the epsilon used for MNIST is too
small. It is better to use an epsilon that is aligned with previous works. For
example in Wong et al., 2018 (https://arxiv.org/pdf/1805.12514.pdf), page 22,
you will find the epsilon used for MNIST and CIFAR.
4. As discussed above, it is probably not fair to compare PER+at with KW. A new
baseline like KW+at should also be considered.
5. For norms other than L infinity norms, solving (5) for getting $d$ can be
time consuming (Algorithm 2). How much additional time does it comparing
to KW?
Overall, I cannot recommend accepting this paper due to its limited theoretical
contribution as well as unconvincing empirical results comparing to previous
methods. I suggest rephrasing some parts of the paper and providing more
experimental results as discussed above.


This paper studies similarity search, which is a crucial enabler of many feature vector-based applications. The problem of similarity search has been extensively studied in the machine learning community. Recent advances of proximity graphs have achieved outstanding performance through exploiting the navigability of the underlying graph structure. In this work, we introduce the annealable proximity graph (APG) method to learn and reshape proximity graphs for efficiency and effective similarity search. APG makes proximity graph edges annealable, which can be effectively trained with a stochastic optimization algorithm. APG identifies important edges that best preserve graph navigability and prune inferior edges without drastically changing graph properties. Experimental results show that APG achieves state-of-the-art results not only by producing proximity graphs with less number of edges but also speeding up the search time by 20-40% across different datasets with almost no loss of accuracy.

This paper suggests an approach for learning how to sparsify similarity search graphs. Graph-based methods currently attain state of the art performance for similarity search, and reducing their number of edges may speed them up even further. The paper suggests a learning framework that uses sample queries in order to determine which edges are more useful for searches, and prune the less useful edges. This is a sensible and potentially useful approach in line with the recent flurry of work on improving algorithms with tool from machine learning.
While I like the overall approach and believe it could work, the experiments seem to have some weaknesses:
1. It is not clear to me why Table 1 contains only UPG and APG with pruning half the edges, without natural pruning baselines like uniformly subsampling the edges by a factor of half, or constructing the graph with half as many edges to begin with. Both of these baseline appear in the plots afterwards, which suggest very similar performance to APG, and it would be interesting to see the numbers side by side. The numbers for UPG and APG alone do not say much: the fact that the number of edges drops by half and the search speed drops by somewhat less than half are inevitable artifacts of the construction. The interesting part is the effect on the accuracy, and its quality is hard to assess without comparison to any baselines.
2. The plots leave the impression that the proposed algorithm does not actually perform that well. It is superior on SIFT, but does not improve performance on GloVe, and is outperformed on Deep1M. This seems to render the textual description of the results somewhat overstated, if I read it right (is it referring only to SIFT?).
In conclusion, while I am optimistic about the paper and the approach, I am tentatively setting my score below the bar in light of the somewhat unsatisfactory experimental performance. The paper would be significantly helped by showing non-negligible improvement on more than one dataset or in more settings.
Other comments:
1. What is NSG? I could not find a spelling out of the abbreviation nor a reference.
2. HNSW-sparse and HNSW-rand are very nearly impossible to tell apart in the plots. I suggest using a clearer visual distinction.
3. "Interestingly, pruning provides the benefits of improved search efficiency" - isn't that the point of pruning?
4. It is curious that using HNSW with R=32 instead of R=64 hurts the performance so much on Deep1M, while it has hardly any effect on SIFT and GloVe, do you perhaps have an explanation for this result?


Neural network out-of-distribution (OOD) detection aims to identify when a model is unable to generalize to new inputs, either due to covariate shift or anomalous data. Most existing OOD methods only apply to classification tasks, as they assume a discrete set of possible predictions. In this paper, we propose a method for neural network OOD detection that can be applied to regression problems. We demonstrate that the hidden features for in-distribution data can be described by a highly concentrated, low dimensional distribution. Therefore, we can model these in-distribution features with an extremely simple generative model, such as a Gaussian mixture model (GMM) with 4 or fewer components. We demonstrate on several real-world benchmark data sets that GMM-based feature detection achieves state-of-the-art OOD detection results on several regression tasks. Moreover, this approach is simple to implement and computationally efficient.

===Summary===
The authors propose to perform out of distribution detection for regression models by fitting a generative model in the feature space of the regression model. An input example is deemed to be out-of-distribution if it has low likelihood under this generative model.
===Overall Assessment===
I recommend that the paper is rejected. There are a number of aspects that need to be improved. You should fix these and resubmit to a future conference.
The paper focuses on the difference between regression and classification tasks and claims that the paper's method addresses an unmet need for OOD for regression. However, both the proposed method and the analysis justifying it are generic enough to be applied to both regression and classification.
The paper handles technical claims far too casually in sec 4 and does not provide sufficient justification that the claims are true.
There are natural baselines, such as using a generative model on the raw input space, that are ignored.
===Comments===
Remark 1 feels to me like it was added for the sake of having more math in the paper, not because it is crucial to the paper's argument.
You remark at various places that existing methods don't naturally generalize from classification to regression. However, you never fully explain why. Also, your proposed method can be applied out-of-the box to classification problems. Your analysis in sec 4 trivially applies to binary classification tasks, and could be naturally extended to multi-class classification where w is not a vector but a num_classes x num_features matrix.
The parallel should be between classification and heteroskedastic regression, since there you have a distribution per example.
The logic in "In-distribution features are intrinsically low dimensional" is insufficient
The connection between section 4 and your proposed method is not particularly precise. You also have lots of technical claims in 4 that are unsubstantiated. For example, you write "this new network will likely have less discarded information than the shallower network". What does 'likely' mean? In what sense are you making an actual technical statement? Each of the subsections in sec 4 has similar issues.
"The CNNs are pre-trained on ImageNet (Denget al., 2009) and the last layer is replaced with a linear layer that produces a single output."
Why did you do this? Did you fine tune or just retrain the top layer?
"For these two baselines, the variance of the forward passes is used as a metric for detecting OOD inputs"
   Can you explain why these are reasonable baselines for OOD? Why no baseline that fits a generative model in input space?
You should cite Ren et al. "Likelihood Ratios for Out-of-Distribution Detection"


Magnitude-based pruning is one of the simplest methods for pruning neural networks. Despite its simplicity, magnitude-based pruning and its variants demonstrated remarkable performances for pruning modern architectures. Based on the observation that the magnitude-based pruning indeed minimizes the Frobenius distortion of a linear operator corresponding to a single layer, we develop a simple pruning method, coined lookahead pruning, by extending the single layer optimization to a multi-layer optimization. Our experimental results demonstrate that the proposed method consistently outperforms the magnitude pruning on various networks including VGG and ResNet, particularly in the high-sparsity regime.

*Summary*
The paper proposes a multi-layer alternative to magnitude-based pruning. The operations entailed in the previous, current, and subsequent layers are treated as linear operations (by omitting any nonlinearities), weights are selected for pruning to minimize the "Frobenius distortion", the Frobenius norm of the difference between products of the (i-1, i, i+1)-layer Jacobians with and without the selected weight. This simplifies to a cost-effective pruning criterion. In spite of the simplistic linear setting assumed for the derivation, results show the criterion prunes better than weight-based methods at unstructured pruning of a variety of modern architectures with CIFAR-10, particularly excelling at higher sparsity.
*Rating*
The paper has some clear positives, particularly:
    + Clear writing and formatting
    + Simple method
    + Good structure for the experimental analysis (with 5x replications!)
However there are a few limitations, noted below; while none is fatal on its own, in total the limitations have led me to recommend "weak reject" currently.
Limitations of the method:
(1) Residual networks: The lack of an explicit strategy for handling residual connections (and the accompanying worsened relative performance) is a notable limitation since residual/skip connections are nearly universal in state-of-the-art large networks. The performance was shown to still be *slightly* better than with magnitude pruning.
(2) Global ranking: Since connections are pruned layerwise, rather than taking the best-k neurons across the entire network at once, I assume that the LAP pruning criterion doesn't scale reasonably across layers. This implies that the method cannot be used to learn network structure. Instead the user must decide the desired number of neurons at each layer.
(3) Structured pruning: There is no mention of pruning entire convolutional kernels or "neurons" at once, so I assume that only individual weights were pruned. Since structured pruning is the simplest way to achieve speedup in network inference (as opposed to merely reduction in model size), how does the LAP criterion perform when adapted for structured pruning, e.g. removing filters/neurons with the best average LAP score?
Limitations of the experiments:
(4) Baselines: While the paper is explicitly focused on an easy to compute replacement for magnitude-based pruning, there are a wide variety of alternative methods available. These vary in complexity, runtime, etc., but they deserve mention and either explicit comparison in the experiments or reasoning to justify the omission of such comparisons.
(5) ImageNet: (Insert obligatory statement about the ubiquity of ImageNet experiments, ...) While it is cliche to request ImageNet experiments and CIFAR-10 is a helpful stand-in, they would be really nice to have.
(6) Activations after non-linearities: While Fig. 3 and the remaining experiments present a reasonable case that the presence of non-linearities doesn't prevent LAP from improving upon magnitude-based pruning, it doesn't resolve the issue either. Whether considering negative values clipped by with ReLU or large magnitude values that are squashed by sigmoid and tanh, the linear-only model is a poor approximation for some unknown fraction of neurons for probably most inputs. Does this mean that LAP is underperforming in those cases? Are those cases sufficiently rare or randomly distributed that they are merely noise? Is there another mechanism at play? In practical terms, how much does the activation rate (positive for ReLU, linear/unsquashed for sigmoid/tanh) vary by neuron? This seems like a reasonably simple thing to compute and incorporate into pruning.
*Notes*
Eq. (4): Is (4) simply the one-step/greedy approximation to the optimization in (3)? If so, it may be helpful to state this explicitly. Also, is $w = W_i[j,k]$? If so, this is useful to explicitly state.
Sec 2.1: Consider noting that the linear-model setup is used to construct the method, but non-linearities are addressed subsequently
Sec 2.2: Is the activation probability p_j used in practice, or is it merely an explanatory device?
pg5: "gradually prune (p/5)%" and marked with a suffix '-seq'"
pg5: note that residual connections are discussed in the experiments?
Tables 3-6: note that these all use CIFAR-10


Automatic neural network discovery methods face an enormous challenge caused by the size of the search space. A common practice is to split this space at different levels and to explore only a fraction of it. On one hand, neural architecture search methods look at how to combine a subset of layers to create an architecture while keeping a predefined number of filters in each layer. On the other hand, pruning techniques take a well known architecture and look for the appropriate number of filters per layer. In both cases, the exploration is made iteratively, training models several times during the search. Inspired by the constraints and advantages of these two approaches, we propose a straight-forward and fast option to find models with improved characteristics. We apply a small set of templates, that have been heuristically and experimentally evaluated, to make a one-shot redistribution of the number of filters in an already existing neural network. When compared to the initial base models we found that the resulting architectures, when trained from scratch, surpass the original accuracy even after been reduced to fit to the original amount of resources. Specifically, we show accuracy improvement for some network-task pairs of up to 5.5%, a reduction of up to 45% in parameters and 60% reduction in memory footprint.

This paper presents a simple methodological study on the effect of the distribution of  convolutional filters on the accuracy of deep convolutional networks on the CIFAR 10 and CIFAR 100 data sets. There are five different kind of distributions studied: constant number of filters, monotonically increasing and decreasing number of filters and convex/concave with a local extremum at the layer in the middle.  For these distributions, the total number of filters is varied to study the trade-off between running-time vs. accuracy, memory vs. accuracy and parameter count vs. accuracy.
Although the paper is purely experimental without any particular theoretical considerations, it presents a few surprising observations defying conventional wisdom:
- The standard method of increasing the number of filters as the number of convolutional nodes is increasing is not the most optimal strategy in most cases.
- The optimal distribution of channels is highly dependent on the network architecture.
- Some network architectures are highly stable with respect to the distribution of channels, while others are very sensitive.
Given that this paper is easy to read and presents interesting insights for the design of convolutional network architectures and challenges mainstream views, I would consider it to be a generally valuable contribution, at least I enjoyed reading it.
Despite the intriguing nature of this paper, there are several weaknesses which make me less enthusiastic about the quality of the paper:
- The experiments are done only on CIFAR-10 and CIFAR-100. These benchmarks are somewhat special. It would be useful to see whether the results also hold for more realistic vision benchmarks. Even if running all the experiments would be costly, I think that at least a small selection should be reproduced on OpenImages or MS-Coco or other more realistic benchmarks to validate the findings of this paper.
- It would be interesting to see whether starting from the best channel distributions, applying MorphNet would end up with different distributions. In general: whether MorphNet would end up with similar distributions automatically.
-  The paper does not clarify how the channel sizes for Inception were distributed, since proper balancing of the 1x1 and more spread out convolutions is a key part of that architecture. This is not clarified in this paper.
- The grammar of the paper is poor, even the abstract is hard to read and interpret.
- The paper presents itself as a methodology for automatically generating the optimal number of channels, while it is more of a one-off experiment and observation than a general purpose method.
Another small technical detail regarding the choice of colors in the diagrams: the baseline distribution and constant distribution are very hard to distinguish. This is especially critical because these are the two best distributions on average. Also the diagrams could benefit from more detailed captions.
The paper presents interesting, valuable experimental findings, but it is not extremely exciting theoretically. Also its practical execution is somewhat lacking. If it contained at least partial results on more realistic data sets, I would vote for strong accept, but in its current form, I find it borderline acceptance-worthy.


We study the role of intrinsic motivation as an exploration bias for reinforcement learning in sparse-reward synergistic tasks, which are tasks where multiple agents must work together to achieve a goal they could not individually. Our key idea is that a good guiding principle for intrinsic motivation in synergistic tasks is to take actions which affect the world in ways that would not be achieved if the agents were acting on their own. Thus, we propose to incentivize agents to take (joint) actions whose effects cannot be predicted via a composition of the predicted effect for each individual agent. We study two instantiations of this idea, one based on the true states encountered, and another based on a dynamics model trained concurrently with the policy. While the former is simpler, the latter has the benefit of being analytically differentiable with respect to the action taken. We validate our approach in robotic bimanual manipulation and multi-agent locomotion tasks with sparse rewards; we find that our approach yields more efficient learning than both 1) training with only the sparse reward and 2) using the typical surprise-based formulation of intrinsic motivation, which does not bias toward synergistic behavior. Videos are available on the project webpage: https://sites.google.com/view/iclr2020-synergistic.

The paper proposes a novel algorithm for encouraging synergistic behavior in multi-agent setups with an intrinsic reward that promotes the agents to work together to achieve states that they cannot achieve individually without cooperation. The paper focuses on a two-agent environment where an approximate forward dynamics model is learnt for each agent, and can be composed sequentially to predict the next environment state given each agent’s action. However, this prediction will be inaccurate if the agent’s affected the environment state in such a way that individual dynamics model cannot predict i.e. synergistic behavior was produced. This prediction error is used as extrinsic reward by the proposed approach, while also having a variant where the true next state is replaced by another approximation of a joint forward model which allows for differentiability of actions with respect to the intrinsic reward. Empirical analysis shows that this intrinsic reward promotes synergetic behavior on two-agent robotic manipulation tasks and achieves better performance that baselines and ablations.
I vote for weak accept as the paper proposes a novel intrinsic reward for promoting synergetic behavior in multi-agent systems, while also demonstrating that such an intrinsic reward can be differentiable if a joint forward dynamics model is approximated in addition to individual forward dynamics models given each agent’s actions. The paper does not show experiments beyond 2 agents and the four robotic manipulation tasks have been shown to work when provided with generic skills as an action space, which requires hand-defining or learning by demonstration.
From the description of the random policy, it is stated that a random policy over skills serves as a sanity check to ensure that the skills do not trivialize the task. This seems to suggest that the extrinsic reward only baseline did not use these skills and was disadvantaged. Some clarification is required here - did the extrinsic reward only baseline use the same skills as the proposed method? If it did, it would obviate the need to have a random policy sanity check.
The paper suggests a baseline for separate arm surprise. In a similar vein, why wasn’t a joint-arm surprise baseline employed, which can basically treat both arms as a single agent?
Synergistic behavior seems hard to achieve for a large number of agents, but the paper does not give insights into whether such an algorithm will work for more than 2 agents. Typically multi-agent systems in prior work have worked with a large number of agents in environments other than robotic manipulation - such experiments may help in strengthening the proposed method.


For an explanation of a deep learning model to be effective, it must provide both insight into a model and suggest a corresponding action in order to achieve some objective. Too often, the litany of proposed explainable deep learning methods stop at the first step, providing practitioners with insight into a model, but no way to act on it. In this paper, we propose contextual decomposition explanation penalization (CDEP), a method which enables practitioners to leverage existing explanation methods in order to increase the predictive accuracy of deep learning models. In particular, when shown that a model has incorrectly assigned importance to some features, CDEP enables practitioners to correct these errors by directly regularizing the provided explanations. Using explanations provided by contextual decomposition (CD) (Murdoch et al., 2018), we demonstrate the ability of our method to increase performance on an array of toy and real datasets.

This paper presents a method intended to allow practitioners to *use* explanations provided by various methods. Concretely, the authors propose contextual decomposition explanation penalization (CDEP), which aims to use explanation methods to allow users to dissuade the model from learning unwanted correlations.
The proposed method is somewhat similar to prior work by Ross et al., in that the idea is to include an explicit term in the objective that encourages the model to align with prior knowledge. In particular, the authors assume supervision --- effectively labeled features, from what I gather --- provided by users and define an objective that penalizes divergence from this. The object that is penalized is $\Beta(x_i, s)$, which is the importance score for feature s in instance $i$; for this they use a decontextualized representation of the feature (this is the contextual decomposition aspect). Although the authors highlight that any differentiable scoring function could be used, I think the use of this decontextualized variant as is done here is nice because it avoids issues with feature interactions in the hidden space that might result in misleading 'attribution' w.r.t. the original inputs.
The main advantage of this effort compared to work that directly penalizes the gradients (as in Ross et al.) is that the method does not rely on second gradients (gradients of gradients), which is computationally problematic. Overall, this is a nice contribution that offers a new mechanism for exploiting human provided annotations. I do have some specific comments below.
- I am not sure I agree with the premise as stated here. Namely, the authors write "For an explanation of a deep learning model to be effective, it must provide both insight into a model and suggest a corresponding action in order to achieve some objective" -- I would argue that an explanation may be useful in and of itself by highlighting how a model came to a prediction. I am not convinced that it need necessarily lead to, e.g., improving model performance. I think the authors are perhaps arguing that explanations might be used to interactively improve the underlying model, which is an interesting and sensible direction.
- This work, which aims to harness user supervision on explanations to improve model performance, seems closely related to work on "annotator rationales" (Zaidan 2007 being the first work on this), but no mention is made of this. "Do Human Rationales Improve Machine Explanations?" by Strout et al. (2019) also seems relevant as a more recent instance in this line of work. I do not think such approaches are necessarily directly comparable, but some discussion of how this effort is situatied with respect to this line of work would be appreciated.
- The experiment with MNIST colors was neat.
- The authors compare their approach to Ross and colleagues in Table 1 but see quite poor results for the latter approach. Is this a result of the smaller batch size / learning rate adjustment? It seems that some tuning of this approach is warranted.
- Figure 3 is nice but not terribly surprising: The image shows that the objective indeed works as expected; but if this were not the case, then it would suggest basically a failure of optimization (i.e., the objective dictates that the image should look like this *by construction*). Still, it's a good sanity check.


Control policies, trained using the Deep Reinforcement Learning, have been recently shown to be vulnerable to adversarial attacks introducing even very small perturbations to the policy input. The attacks proposed so far have been designed using heuristics, and build on existing adversarial example crafting techniques used to dupe classifiers in supervised learning. In contrast, this paper investigates the problem of devising optimal attacks, depending on a well-defined attacker's objective, e.g., to minimize the main agent average reward. When the policy and the system dynamics, as well as rewards, are known to the attacker, a scenario referred to as a white-box attack, designing optimal attacks amounts to solving a Markov Decision Process. For what we call black-box attacks, where neither the policy nor the system is known, optimal attacks can be trained using Reinforcement Learning techniques. Through numerical experiments, we demonstrate the efficiency of our attacks compared to existing attacks (usually based on Gradient methods). We further quantify the potential impact of attacks and establish its connection to the smoothness of the policy under attack. Smooth policies are naturally less prone to attacks (this explains why Lipschitz policies, with respect to the state, are more resilient). Finally, we show that from the main agent perspective, the system uncertainties and the attacker can be modelled as a Partially Observable Markov Decision Process. We actually demonstrate that using Reinforcement Learning techniques tailored to POMDP (e.g. using Recurrent Neural Networks) leads to more resilient policies.

This paper investigates the design of adversarial policies (where the action of the adversarial agent corresponds to a perturbation in the state perceived by the primary agent). In particular it focuses on the problem of learning so-called optimal adversarial policies, using reinforcement learning.
I am perplexed by this paper for a few reasons:
1)	What is the real motivation for this work?  The intro argues “casting optimal attacks is crucial when assessing the robustness of RL policies, since ideally, the agent should learn and apply policies that resist *any* possible attack”.   If the goal is to have agents that are robust to *any* attacks, then they cannot be robust just to so-called optimal attacks.  And so what is really the use of learning so-called optimal attacks?
2)	The notion itself of “optimal” attack is not clear.  The paper does not properly discuss this.  It quickly proposes one possible definition (p.4): “the adversary wishes to minimize the agent’s average cumulative reward”.  This is indeed an interesting setting, and happens to have been studied extensively in game-theoretic multi-agent systems, but the paper does not make much connection with that literature (apart from a brief mention at bottom of p.2 / top of p.3), so it’s not clear what is new here compared to this.   It’s also not discussed whether it would ever be worthwhile considering other notions of optimality for the adversary, and what would be the properties of those.
So overall, while I find the general area of this work to be potentially interesting, the current framing is not well motivated enough, and not sufficiently differentiated from other work in robust MDPs and multi-agent RL to make a strong contribution yet.
More minor comments:
-	P.3: “very different setting where the adversary has a direct impact on the system” => Clarify what are the implications of this in terms of framework, theory, algorithm.
-	P.4: You assume a valid Euclidean distance for the perturbed state.  Is this valid in most MDP benchmarks?  How is this implemented for the domains in the experiments?  What is the action space considered? Do you always assume a continuous action space for the attacker?
-	P.5: “we can simply not maintain distributions over actions” -> Why not?  Given the definition of perturbation, this seems feasible.
-	P.5:  Eqn 4 is defined for a very specific adversarial reward function. Did you consider others?  Is the gradient always easy to derive?
-	P.6: Eqn (5) & (6): What is “R” here?
-	P.7: Figure 1, top right plot. Seems here that the loss is above 0 for small \epsilon.  Is this surprising?  Actually improving the policy?
-	P.7: What happens if you consider even greater \epsilon?  I assume the loss is greater.  But then the perturbation would be more detectable?  How do you think about balancing those 2 requirements of adversarial attacks?  How should we formalize detectability in this setting?
-	Fig.2: Bottom plots are too small to read.
-	Sec.6:  Can you compare to multi-agent baselines, e.g. Morimoto & Doya 2005.
-	P.8: “We also show that Lipschitz policies have desirable robustness properties.” Can you be more specific about where this is shown formally?  Or are you extrapolating from the fact that discrete mountain car suffers more loss than continuous mountain car?  I would suggest making that claim more carefully.


User generated content contains opinionated texts not only in dominant languages (like English) but also less dominant languages( like Amharic). However, negation handling techniques that supports for sentiment detection is not developed in such less dominant language(i.e. Amharic). Negation handling is one of the challenging tasks for sentiment classification. Thus, this work builds negation handling schemes which enhances Amharic Sentiment classification. The proposed Negation Handling framework combines the lexicon based approach and character ngram based machine learning model. The performance of framework is evaluated using the annotated Amharic News Comments. The system is outperforming the best of all models and the baselines by an accuracy of 98.0. The result is compared with the baselines (without negation handling and word level ngram model).

This paper is quite difficult to read. The figures are pixelated. There is almost no organization to the text of the work.
Descriptions are imprecise and lax: for example, "We apply basic preprocessing on Amharic News Comments. These include normalization of Amharic script symbols, tokenization, stop word removal, punctuation mark removal and so on. Amharic writing system is expressed using only consonants. To handle the features of the language is very challenging. We require conversion of Amharic scripts to consonant-vowel form. Particularly, before performing negation handling and stemming, the algorithm converts each Amharic word to its consonant vowel form." What kind of normalization? What kind of tokenization? Did you use outside tools? If so, cite them or refer to the code. If not, then please explain your method more thoroughly. Don't assume all your readers know what consonant-vowel form is. "and so on" is not appropriate. You should explain what the so on is in a paper submitted to a conference. Don't bother telling us the features of the language are challenging. Just tell us how you did it.
Switching between words like "cue" and "clue"
Even the font of the text changes part way through the paper.
Figure 2 is almost entirely uninformative.
The algorithms are poorly displayed and nearly unreadable.
Experiments are too limited.