Real-world machine learning applications often have complex test metrics, and may have training and test data that follow different distributions. We propose addressing these issues by using a weighted loss function with a standard convex loss, but with weights on the training examples that are learned to optimize the test metric of interest on the validation set. These metric-optimized example weights can be learned for any test metric, including black box losses and customized metrics for specific applications. We illustrate the performance of our proposal with public benchmark datasets and real-world applications with domain shift and custom loss functions that balance multiple objectives, impose fairness policies, and are non-convex and non-decomposable.

This proposal learns training data weights that optimize any given test metric.
They do this by learning both a weighting function and a classifier. They
iteratively train an ensemble of K {weighting_k,classifier} pairs, and
selecting the pair that presents the best best metric-of-interest value over
the test set (over all iterations) as the final output.
The paper provides a large set of references that I found useful, and is
clearly written.
Each training iteration of their MOEW algorithm first optimizes classifiers
over all training examples, for each of the K sample weightings.  Each
weighting gives rise to one converged classifier, whose metric-of-interest is
evaluated on the validation set.  Given K sets of weighting functions and their
metric-of-interest values, new parameters for the weighting functions are
generated.
The weighting function choice is a simple function, based on a linear transform
of autoencoder features, a normalization factor, and factor to account for
differing label frequencies in training and test data.  The low-dimensional
auto-encoding of {data,label} pairs is trained once, using the training data.
The weighting function parameters are a linear transformation matrix. They
also have a sigmoid non-linearity in their weighting function.
If I understand correctly, their approach seems to generalize the importance
sampling methods that they reference by using a nonlinear combination of autoencoder
features.  They reference importance sampling methods that, for example, use
Gaussian kernel basis functions to model training/test example densities.
While they do provide extensive references to previous approaches, I would have
enjoyed a clearer explanation of main differences between their weighting
function and ones that have previously been used in importance sampling.
Their main contribution seems to be the procedure (Alg. 2) used to updated the
weighting function parameters.  They first fit a model that predicts
metric-of-interest values given all previous weighting function parameters used
in the algorithm.  Then they use this model to generate the next set of
weighting function parameters.  Their method adapts a Gaussian Process
Upper-Confidence Bound to batch-wise processing.  Within-batch and between
batch exploration of the weighting function parameters is controlled by two
parameters, which they held fixed at values corresponding to +/-1 for a normal
distribution.
Only the validation set need be iid with the test set, so their method seems
quite general.
Their MNIST results use very simple networks to show that learning weighting
functions has the most benefit when classifier networks are severely
under-parameterized.  Their wine price example addresses the choice of
embedding dimension, and they found their error metric decreased from 52% with
uniform weighting, to around 46% as they approached ~10 dimensions in the
autoencoded {data,label} representation of the training data.  They then use a
small crime dataset with a complicated test metric measuring *fairness*, based
on dividing the dataset into 4 quantiles based on white population. Using
MOEW they could improve the fairness metric with little effect on the accuracy
metric.  They could also preset thresholds to achieve very good fairness on the
training data and use MOEW to maximize the accuracy metric, which resulting
in improving both fairness and accuracy compared to uniform sample weighting.
For spam blocking and web page quality, I would like to see a little more
interpretation of their results. They again show improvements using MOEW to
provide weights for training data.  This time, Table 2 presents a comparison
with importance sampling, but the methodology they used for importance sampling
is not well described.  I'd like to understand where this improvement came
from. What model/procedure was used for importance weighting? Gaussian rbfs?
Do the authors attribute the dramatic improvement for MOEW for Spam Blocking
simply to having a more flexible sample distribution function?  Or is it mainly
due to their adapting to the test metric?  Then for Web Page Quality, would it
be correct to conclude that the old and new web pages (training vs
validation/test) are actually fairly similary distributed?
What values of B and K are reasonable values in Alg. 1?  When applied to larger
problems, where do the authors feel the bottlenecks in Algs 1 and 2 will lie?
Do the authors find that retuning the classifier for different weighted samples
to take a lot of time?
Pros:
 - tests on several datasets
 - seems fairly generally applicable.

Cons:
 - To reproduce, I guess I'd need to adapt existing GP-UCB code from python or
   C++, and I'm not sure how easy this would be. Releasing code to reproduce
   the results might be nice.
 - Better understanding of differences wrt. typical Importance Sampling methods
   would nice.


An increasing number of neural memory networks have been developed, leading to the need for a systematic approach to analyze and compare their underlying memory capabilities. Thus, in this paper, we propose a taxonomy for four popular dynamic models: vanilla recurrent neural network, long short-term memory, neural stack and neural RAM and their variants. Based on this taxonomy, we create a framework to analyze memory organization and then compare these network architectures. This analysis elucidates how different mapping functions capture the information in the past of the input, and helps to open the dynamic neural network black box from the perspective of memory usage. Four representative tasks that would fit optimally the characteristics of each memory network are carefully selected to show each network's expressive power. We also discuss how to use this taxonomy to help users select the most parsimonious type of memory network for a specific task. Two natural language processing applications are used to evaluate the methodology in a realistic setting.

I really liked this paper and believe it could be useful to many practitioners of NLP, conversational ML and sequential learning who may find themselves somewhat lost in the ever-expanding field of dynamic neural networks.
Although the format of the paper is seemingly unusual (it may feel like reading a survey at first), the authors propose a concise and pedagogical presentation of Jordan Networks, LSTM, Neural Stacks and Neural RAMs while drawing connections between these different model families.
The cornerstone of the analysis of the paper resides in the taxonomy presented in Figure 5 which, I believe, should be presented on the front page of the paper. The taxonomy is justified by a thorough theoretical analysis which may be found in appendix.
The authors put the taxonomy to use on synthetic and real data sets. Although the data set taxonomy is less novel it is indeed insightful to go back to a classification of grammatical complexity and structure so as to enable a clearer thinking about sequential learning tasks.
An analysis of sentiment analysis and question answering task is conducted which relates the properties of sequences in those datasets to the neural network taxonomy the authors devised. In each experiment, the choice of NN recommended by the taxonomy gives the best performance among the other elements presented in the taxonomy.
Strength:
o) The paper is thorough and the appendix presents all experiments in detail.
o) The taxonomy is clearly a novel valuable contribution.
o) The survey aspect of the paper is also a strength as it consolidates the reader's understanding of the families of dynamic NNs under consideration.
Weaknesses:
o) The taxonomy presented in the paper relies on an analysis of what the architectures can do, not what they can learn. I believe the authors should acknowledge that the presence of Long Range Dependence in sequences is still hard to capture by dynamic neural networks (in particular RNNs) and that alternate analysis have been proposed to understand the impact of the presence of such Long Range Dependence in the data on sequential learning. I believe that mentioning this issue along with older (http://ai.dinfo.unifi.it/paolo/ps/tnn-94-gradient.pdf) and more recent (e.g. http://proceedings.mlr.press/v84/belletti18a/belletti18a.pdf and https://arxiv.org/pdf/1803.00144.pdf) papers on the topic is necessary for the paper to present a holistic view of the matter at hand.
o) The arguments given in 5.2 are not most convincing and could benefit from a more thorough exposition, in particular for the sentiment analysis task. It is not clear enough in my view that it is true that "since the goal is to classify the emotional tone as either 1 or 0, the specific contents of the text are not very important here". One could argue that a single word in a sentence can change its meaning and sentiment.
o) The written could be more polished.
As a practitioner using RNNs daily I find this paper exciting as an attempt to conceptualize both data set properties and dynamic neural network families. I believe that the authors should address the shortcomings I think hinder the paper's arguments and exposition of pre-existing work on the analysis of dynamic neural networks.


MULTI-CLASS CLASSIFICATION WITHOUT MULTI- CLASS LABELS
This work presents a new strategy for multi-class classification that requires no class-specific labels, but instead leverages pairwise similarity between examples, which is a weaker form of annotation. The proposed method, meta classification learning, optimizes a binary classifier for pairwise similarity prediction and through this process learns a multi-class classifier as a submodule. We formulate this approach, present a probabilistic graphical model for it, and derive a surprisingly simple loss function that can be used to learn neural network-based models. We then demonstrate that this same framework generalizes to the supervised, unsupervised cross-task, and semi-supervised settings. Our method is evaluated against state of the art in all three learning paradigms and shows a superior or comparable accuracy, providing evidence that learning multi-class classification without multi-class labels is a viable learning option.

This paper proposed how to learn multi-class classifiers without multi-class labels. The main idea is shown in Figure 2, to regard the multi-class labels as hidden variables and optimize the likelihood of the input variables and the binary similarity labels. The difference from existing approaches is also illustrated in Figure 1, namely existing methods have binary classifiers inside multi-class classifiers while the proposed method has multi-class classifiers inside binary classifiers. The application of this technique to three general problem settings is discussed, see Figure 3.
Clarity: Overall, it is very well written. I just have two concerns.
First, the authors didn't discuss the underlying assumption of the proposed method except the additional independence assumption. I think there should be more underlying assumptions. For example, by the definition P(S_{i,j}=0 or 1|Y_i,Y_j) and the optimization of L(theta;X,S), does the "cluster assumption" play a role in it? The cluster assumption is popular in unsupervised/semi-supervised learning and metric learning where the X part of training data is in a form of pairs or triples. However, there is no such an assumption in the original supervised multi-class learning. Without figuring out the underlying assumptions, it is difficult to get why the proposed method works and when it may fail.
Second, there are too many abbreviations without full names, and some of them seem rather important such as KLD and KCL. I think full names of them should be given for the first time they appear. This good habit can make your audience more broad in the long run.
Novelty: As far as I know, the proposed approach is novel. It is clear that Section 3 is original. However, due to the writing style, it is hard to analyze which part in Section 4 is novel and which part is already known. This should be carefully revised in the final version. Moreover, there was a paper in ICML 2018 entitled "classification from pairwise similarity and unlabeled data", in which binary classifiers can be trained strictly following ERM without introducing the cluster assumption. The same technique can be used for learning from pairwise dissimilarity and unlabeled data as well as from pairwise similarity and dissimilarity data. I think this paper should be included in Section 2, the related work.
Significance: I didn't carefully check all experimental details but the experimental results look quite nice and promising. Given the fact that the technique used in this paper can be applied to many different tasks in machine learning ranging from supervised learning to unsupervised learning, I think this paper should be considered significant.
Nevertheless, I have a major concern as follows. In order to derive Eq. (2), the authors imposed an additional independence assumption: given X_i and X_j, S_{i,j} is independent of all other S_{i',j'}. Hence, Eqs. (2) and (3) approximately hold instead of exactly hold. Some comments should be given on how realistic this assumption is, or equivalently, how close (1) and (3) are. One more minor concern: why P(X) appears in (1) and then disappears in (2) and (3) when Y is marginalized?


Recently Neural Architecture Search (NAS) has aroused great interest in both academia and industry, however it remains challenging because of its huge and non-continuous search space. Instead of applying evolutionary algorithm or reinforcement learning as previous works, this paper proposes a Direct Sparse Optimization NAS (DSO-NAS) method. In DSO-NAS, we provide a novel model pruning view to NAS problem. Specifically, we start from a completely connected block, and then introduce scaling factors to scale the information flow between operations. Next, we impose sparse regularizations to prune useless connections in the architecture. Lastly, we derive an efficient and theoretically sound optimization method to solve it. Our method enjoys both advantages of differentiability and efficiency, therefore can be directly applied to large datasets like ImageNet. Particularly, on CIFAR-10 dataset, DSO-NAS achieves an average test error 2.84%, while on the ImageNet dataset DSO-NAS achieves 25.4% test error under 600M FLOPs with 8 GPUs in 18 hours.

Summary:
This paper proposes Direct Sparse Optimization (DSO)-NAS, which is a method to obtain neural architectures on specific problems, at a reasonable computational cost.
The main idea is to treat all architectures as a Directed Acyclic Graph (DAG), where each architecture is realized by a subgraph. All architectures in the search space thus share their weights, like ENAS (Pham et al 2018) and DARTS (Liu et al 2018a). The DAG’s edges can be pruned via a sparsity regularization term. The optimization objective of DSO-NAS is thus:
Accuracy + L2-regularization(W) + L1-sparsity(\lambda),
where W is the shared weights and \lambda specifies which edges in the DAG are used.
There are 3 phases of optimization:
1. All edges are activated and the shared weights W are trained using normal SGD. Note that this step does not involve \lambda.
2. \lambda is trained using Accelerated Proximal Gradient (APG, Huang and Wang 2018).
3. The best architecture is selected and retrained from scratch.
This procedure works for all architectures and objectives. However, DSO-NAS further proposes to incorporate the computation expense of architectures into step (2) above, leading to their found architectures having fewer parameters and a smaller FLOP counts.
Their experiments confirm all the hypotheses (DSO-NAS can find architectures, having small FLOP counts, having good performances on CIFAR-10 and ImageNet).
Strengths:
1. Regularization by sparsity is a neat idea.
2. The authors claim to be the first NAS algorithm to perform direct search on ImageNet. Honestly, I cannot confirm this claim (not sure if I have seen all NAS papers out there), but if it is the case, then it is impressive.
3. Incorporating architecture costs into the search objective is nice. However, this contribution seems to be orthogonal to the sparsity regularization, which, I suppose, is the main point of the paper.
Weaknesses:
1. Some experimental details are missing. I’m going to list them here:
- Was the auxiliary tower used during the training of the shared weights W?
- Figure 4 does not illustrate M=4 and N=4, e.g. which operation belongs to which layer?
- Did the experiments on CIFAR-10 and ImageNet use the cosine learning rate schedule [1]? If or if not, either way, you should specify it in a revised version of this paper, e.g. did you use the cosine schedule in the first 120 steps to train the shared parameters W, did you use it in the retraining from scratch?
- In Section 3.3, it is written that “The sparse regularization of \lambda induces great difficulties in optimization”. This triggers my curiosity of which difficulty is it? It would be nice to see this point more elaborated, and to see ablation study experiments.
2. Missed citation: MnasNet [2] also incorporates the cost of architectures in their search process. On ImageNet, your performance is similar to theirs. I think this will be a good comparison.
3. The paper has some grammatical errors. I obviously missed many, but here are the one I found:
- Section 3.3: “Different from pruning, which the search space is usually quite limited”. “which” should be “whose”?
- Section 4.4.1: “DSO-NAS can also search architecture [...]”  -> “DSO-NAS can also search for architectures [...]”
References.
[1] SGDR: Stochastic Gradient Descent with Warm Restarts. https://arxiv.org/pdf/1608.03983.pdf
[2] MnasNet: Platform-Aware Neural Architecture Search for Mobile. https://arxiv.org/pdf/1807.11626.pdf


While deep neural networks have achieved remarkable successes, they suffer the well-known catastrophic forgetting issue when switching from existing tasks to tackle a new one. In this paper, we study continual learning with deep neural networks that learn from tasks arriving sequentially. We first propose an approximated multi-task learning framework that unifies a family of popular regularization based continual learning methods. We then analyze the weakness of existing approaches, and propose a novel regularization method named "Activity Regularization" (AR), which alleviates forgetting meanwhile keeping model's plasticity to acquire new knowledge. Extensive experiments show that our method outperforms state-of-the-art methods and effectively overcomes catastrophic forgetting.

This paper proposes an approach to mitigate catastrophic forgetting in supervised learning by regularizing activations. The paper views previous techniques (EWC, SI, and GEM) under a multi-task learning lens, and then proposes an additional loss term to minimise the KL between activations from previous and current models, on previous tasks - this is based on a memory which stores some previous samples and their corresponding activations.
I think it is a simple and intuitive approach and a well-written paper. Unfortunately I have a number of concerns that I think preclude publication in the current state.
First, in terms of related work, I believe this is very similar to Learning without forgetting (LwF), with the difference that the KL-divergence is computed on samples kept from the previous tasks. This is briefly mentioned in the paper, but I think it needs to be made more explicit, and LwF should be a baseline in the experiments to clearly indicate the benefit of keeping this data. There is also a relationship to EWC: given the connection between the Fisher information and KL, it can be viewed as minimising the KL divergence in parameter space, rather than in activation space (which is the case here). Also note that EWC uses the true Fisher rather than the empirical, contrary to the derivation in equation (2).
There are also a number of papers that haven’t been cited in the related work [1][2][3][4].
Second, I think the motivation in Section 3.1 could be more convincing. Most importantly, it’s not clear to me that the decision boundary *shouldn’t* change for previously misclassified examples, as this could be an opportunity for backwards transfer.
Further, I don’t think the point in the last paragraph about having a small data portion is relevant, since they are from the same data distribution, and we would expect misclassified samples to be in the same (low) frequency in Fisher estimation as overall. I think the point of this paragraph is just that it is important to consider the entire predictive distribution of previous tasks rather than the probability of the correct class, so this should be stated more clearly and then justified.
Finally, I think the experimental justification could be improved as well. Beyond permuted MNIST (which it has been argued is not as useful as other baselines [4]), only the final performance on split notMNIST / CIFAR-100 is reported. Some comments and questions:
- The accuracies of EWC (and possibly SI) in the table are worse than reported in previous work (eg. [1]), so I think this needs to be examined.
- What is the fine-tuning baseline (I don't believe it is actually clearly defined)? How can it be so low in figure 2a but better in 2b?
- I think plots over time (performance on all tasks) would be much more useful than the final performance in Table 2 and Fig 2.
- Errors and error bars would be beneficial for all results.
- Table 1 should also include the references provided.
Some other comments and questions:
- Compared to eqn (2), eqn (6) is missing the ½ constant.
- Typos in section 5.3: "SI performs better than SI", and VAR instead of SAR.
- Section 2, unclear of meaning of "coined with the likelihood" (should this be “coincide”?)
- The first line should be “Humans have the ability to learn...” In general, I think the introduction could use another proofread for grammar and readability as I saw a few minor things.
[1] Nguyen, Cuong V., et al. "Variational Continual Learning." ICLR, 2018.
[2] Schwarz, Jonathan, et al. "Progress & Compress: A scalable framework for continual learning." ICML, 2018.
[3] Shin, Hanul, et al. "Continual learning with deep generative replay." NIPS, 2017.
[4] Farquhar, Sebastian, and Yarin Gal. "Towards Robust Evaluations of Continual Learning." arXiv, 2018.


In this paper, we investigate mapping the hyponymy relation of WORDNET to feature vectors. We aim to model lexical knowledge in such a way that it can be used as input in generic machine-learning models, such as phrase entailment predictors. We propose two models. The first one leverages an existing mapping of words to feature vectors (fastText), and attempts to classify such vectors as within or outside of each class. The second model is fully supervised, using solely WORDNET as a ground truth. It maps each concept to an interval or a disjunction thereof. On the first model, we approach, but not quite attain state of the art performance. The second model can achieve near-perfect accuracy.

This paper reads like the first thoughts and experiments of a physicist or mathematician who has decided to look at word representations and hyponymy. I mean that in both the positive and negative ways that this remark could be construed. On the positive side, it provides an interesting read, with a fresh perspective, willing to poke holes in rather than accepting setups that several previous researchers have used.  On the negative side, though, this paper can have an undue, work-from-scratch mathiness that doesn't really contribute insight or understanding, and the current state of the work is too preliminary. I think another researcher interested in this area could benefit from reading this paper and hearing the perspective it presents. Nevertheless, there just isn't sufficient in the way of strong, non-trivial results in the current paper to justify conference acceptance.
Quality:
 - Pro
   o Everything is presented in a precise formalized fashion. The paper has interesting remarks and perspectives. I appreciate that the authors not only did find most existing work on modeling hyponymy but provide a detailed and quite insightful discussion of it.  (A related paper from overlapping authors to papers you do cite that maybe should have been included is Chang et al. https://arxiv.org/abs/1710.00880 – which is a bit different in trying to learn hyponyms from text not WordNet, but still clearly related.)
  -Con
   o There just isn't enough here in the way of theoretical or experimental results. In the end, two "methods" of hyponymy modeling are presented: one is a simple logistic regression, which is estimated separately PER WORD for words with 10 or more hyponyms. This performs worse than the methods of several recent papers that the author cites. The other is a construction that shows that any tree can be embedded by representing nodes as ranges of the real line. This is true, but trivial. Why don't ML/NLP researchers do this? It's because they want a representation that doesn't only represent the ISA hierarchy but also other aspects of word meaning such as meaning similarity and dimensions of relatedness. Furthermore, in general they would like to learn these representations from data rather than hand-constructing it from an existing source like WordNet. For instance, simply doing that gives no clear way to add other words not in wordnet into the taxonomy. This representation mapping doesn't really give any clear advantage beyond just looking up hyponymy relationships in wordnet when you need them.
Clarity:
 - Pro
   o The paper is in most respects clearly written and enjoyable to read.
 - Con
   o The mathematical style and precision has it's uses, but sometime it just seemed to make things harder to follow. Referring to things throughout as "Property k" – even though some of those properties were given names when first introduced – left me repeatedly flicking up and down through the PDF to refresh myself on what claim was being referred to without any apparent need....
Originality:
 - Pro
   o There is certainly originality of perspective. The authors make some cogent observations on how other prior work has been naive about adopted assumptions and as to what it has achieved (e.g., in the discussion at the start of section 5.1).
 - Con
   o There is not really significant originality of method. The logistic regression model is nothing but straightforward. (It is also highly problematic in learning a separate model for each word with a bunch of hyponyms. This both doesn't give a model that would generalize to novel words or ones with few hyponyms.) Mapping a tree to an interval is fairly trivial, and besides this is just a mapping of representations, it isn't learning a good representation as ML people (or ICLR people) would like. The idea that you can improve recall by using a co-product (disjunction) of intervals is cute, though, I admit. Nice.
Significance
 - Con
   o I think this work would clearly need more development, and more cognizance of the goals of generalizable representation learning before it would make a significant contribution to the literature.
Other:
 - p.1: Saying about WordNet etc., "these resources have a fundamentally symbolic representation, which can not be readily used as input to neural NLP models" seems misplaced when there is now a lot of work on producing neural graph embeddings (including node2vec, skip-graphs, deepwalk, etc.). Fundamentally, it is just a bad argument: It is no different to saying that words have a fundamentally symbolic representation which cannot be readily used as input to neural NLP models, but the premise of the whole paper is already that we know how to do that and it isn't hard through the use of word embeddings.
 - p.2: The idea of words and phrases living in subset (and disjointness etc.) relationships according to denotation is the central idea of Natural Logic approaches, and these might be cited here. There are various works, some more philosophical. A good place to start might be: https://nlp.stanford.edu/pubs/natlog-iwcs09.pdf
 - p.2: The notions of Property 1 and 2 are just "precision" and "recall", terms the paper also uses. Do we gain from introducing the names "Property 1" and "Property 2" for them? I also felt that I wouldn't have lost anything if Property 3 was just the idea that hyponymy is represented as vector subspace inclusion.
 - p.2: fn.2: True, but it seems fair to more note that cosine similarity is very standard as a word/document similarity measure, not for modeling hyponymy, for this reason.
 - p.4: Below the equation, shouldn't it be Q(w', w) [not both w'] and then Q(w', w) and not the reverse? If not, I'm misunderstanding.


Various methods of measuring unit selectivity have been developed in order to understand the representations learned by neural networks (NNs). Here we undertake a comparison of four such measures on AlexNet, namely, localist selectivity Bowers et al. (2014), precision (Zhou et al., 2015, class-conditional mean activity selectivity CCMAS; Morcos et al. (2018), and a new measure called top-class selectivity. In contrast with previous work on recurrent neural networks (RNNs), we fail to find any 100% selective 'localist units' in AlexNet, and demonstrate that the precision and CCMAS measures provide a much higher level of selectivity than is warranted, with the most selective hidden units only responding strongly to a small minority of images from within a category. We also generated activation maximization (AM) images that maximally activated individual units and found that under (5%) of units in fc6 and conv5 produced interpretable images of objects, whereas fc8 produced over 50% interpretable images. Furthermore, the interpretable images in the hidden layers were not associated with highly selective units. These findings highlight the problem with current selectivity measures and show that new measures are required in order to provide a better assessment of learned representations in NNs. We also consider why localist representations are learned in RNNs and not AlexNet.

Summary - This paper analyzes the selectivity of individual units in CNNs. The authors analyze existing techniques such as precision selectivity, class-conditional mean activity selection and localist sensitivity. These methods are analyzed in the context of AlexNet and ImageNet. The authors also use Activation Maximization (AM) techniques for visualizing single-unit representations in CNNs.
Paper strengths
- The authors have minutely examined each of the metrics and the underlying assumptions they make.
    - Example - Number of images used for computing the precision threshold in Zhou et al., 2014; The wrongly stated range of CCMAS [0, 1]. Considering the second highest CCMAS class is a good way of handling multiple classes that activate a single unit.
- The results of this paper are surprising compared to existing work. The authors have made a surprising discovery and done a good job of both presenting it well and experimentally validating it. The paper raises interesting questions and this should inspire future work in understanding networks.
- Figure 2 is insightful - It compares the various different interpretations of selectivity for a single unit in fc6. It shows how the mean activating class and the maximally activating class can be semantically very different. It also shows that despite the high precision and CCMAS score, the unit cannot be labelled as a detector for the single concept "custard apple". More such results are presented in the Appendix (e.g. Fig A6)
- The human study in Section 3.3 is a good way to evaluate the generated AM images.
Paper weaknesses
- One of the major weaknesses of this paper is that it uses only ImageNet images to evaluate the units. As this is limited to 1000 classes, the authors cannot probe other visual concepts such as color, texture, materials for the units. As an example, Network dissection (Zhou et al., 2017) proposes a dataset called Broden which has many diverse sets of visual concepts labeled. This paper focuses only on one definition of selectivity - selecting objects. This should be made explicit and the authors have not done a good job of clarifying this assumption or showing that it exists.
- All of the analysis is limited to AlexNet. With modern architectures that use residual/skip connections, it is not clear how well this analysis will generalize. It is an open question if the authors work overfits to AlexNet.
- The jitterplots are hard to understand especially if there are many overlapping "dots" (samples). Since the y-axis values are not really meaningful anyway, using a histogram to see how many samples have a particular activation value is easier. A possible suggestion for Figure 2(a): split into two parts - 1) histogram of all samples; 2) histogram of the highest mean activating class.
- The organization of the paper could be improved. The sections in the paper are not well connected.


VISUAL EXPLANATION BY INTERPRETATION: IMPROVING VISUAL FEEDBACK CAPABILITIES OF DEEP NEURAL NETWORKS
Interpretation and explanation of deep models is critical towards wide adoption of systems that rely on them. In this paper, we propose a novel scheme for both interpretation as well as explanation in which, given a pretrained model, we automatically identify internal features relevant for the set of classes considered by the model, without relying on additional annotations. We interpret the model through average visualizations of this reduced set of features. Then, at test time, we explain the network prediction by accompanying the predicted class label with supporting visualizations derived from the identified features. In addition, we propose a method to address the artifacts introduced by strided operations in deconvNetbased visualizations. Moreover, we introduce an8Flower, a dataset specifically designed for objective quantitative evaluation of methods for visual explanation. Experiments on the MNIST, ILSVRC12, Fashion144k and an8Flower datasets show that our method produces detailed explanations with good coverage of relevant features of the classes of interest.

Summary: the paper proposes a method for Deep Neural Networks (DNN) that identifies automatically relevant features of the set of the classes, enriching the predictions made with the visual features that contributed to that class, supporting, thus, interpretation (understanding what the model has learned) and explanation (justification of the predictions/classifications made by the model). This scheme does not rely on additional annotations, like earlier techniques do.
The contributions of this paper are relevant to, I would say, a large segment of the AI community, since interpretability and explainability of AI (XAI) is the focus of many current works in the area, and there are still many unresolved issues. I consider this paper suitable for ICLR 2019, in particular, it fits the call for papers topic “visualization or interpretation of learned representations”.
The authors also present a new dataset (am8Flower) that can be used by the community for future evaluations of explanation methods for DNN. From my point of view, this is a significant contribution, since there is a lack of datasets that can be used for evaluation.
The authors motivate properly the need for this research/study, addressing the main weakness of the two more common strategies for interpreting DNN, (1) manually inspecting visualizations of every single filter or (2) comparing the internal activations produced by a given model w.r.t. a dataset with pixel-wise annotations of possibly relevant concepts.
I would encourage the authors to write the limitations and weakness of their proposal w.r.t. similar approaches they reviewed. I am aware that the space is limited, but in p.8, section 4.3, when Table 1 is introduced and the authors confirm that their proposal has higher IoU than other methods, the authors could explain, in brief, what are the weaknesses of their method w.r.t. the other approaches analyzed.
Another clarification concerns the initialization of input parameters, such as sparsity; e.g., p.6 sparsity is initialized with 10 for all datasets, why? How has this value been selected and how sensitive is the performance regarding variations of this value?
Once again, I know that the space is limited, but I would like to be able to see some of the figures better (since this is an essential part of the paper). The additional material complements very well the paper and shows larger figures, but I think that the paper itself should be self-sufficient, and figures like Fig. 5 should be enlarged so it is easier to see some details.
Just a concern or something that I quite did not understand about one of the arguments the authors use to justify the evaluation carried out: the authors claim that they want to avoid the subjectivity introduced by humans (citing Gonzalez-Garcia et al. 2017), and prefer to avoid user studies, presenting a more objective approach in their evaluation. Ok, but then, the analysis presented in, for example, page 7, is based mainly in their interpretation of the results, a qualitative analysis of the images (we can see fur patterns, this and that, etc.). So aren’t they interpreting the results obtained as users? So after all, aren’t the visual explanations and feedback intended for users? Why should we claim that we want to avoid the subjectivity introduced by humans in the evaluation when the method proposed here is actually going to be used by users –with their inherent subjectivity? I do not mean that the evaluation carried out is not interesting per se, but it could be motivated differently, or it could be complemented later on with future user studies (that would make an interesting addition to the paper). Moreover, I also wonder whom the authors see as intended users for the proposed scheme.
Small comments:
P.1 “useful insights on the internal representations”  insights into the internal representations.
P. 2: space needed in “back-propagation methods.Third,”
P. 3: Remove “s” in verb (plural authors): “Similarly, Bach et al. (2015) decomposes the classification”  decompose or decomposed
P.3: n needed “Chattopadhyay et al. (2018) exteded”  extended
P.3: “This saliency-based protocol assume that”  protocol assumes
P.3: “highlighted by the the explanations”  remove one “the”
P. 5: “space. As as result we get”  remove one “as”
P. 5: “and compensate this change”  compensate for this change
P. 6: “In this experiment we verify”  In this experiment, we verify
P. 6: “To this end, given a set of identified features we”  To this end, given a set of identified features, we
P. 6: “Note that the OnlyConv method, makes the assumption”  remove “,” after method
P. 7: “In order to get a qualitative insight of the type of”  insight into the
P. 7: I would write siamese and persian cat with capital “S” and “P” (Siamese, Persian)
P. 7: others/ upper “Some focus on legs, covered and uncovered, while other focus on the upped body part.”  while others focus on the upper body part
P. 7: “These visualizations answers the question”  answer
P. 7:  “In this section we assess”  In this section, we
P. 7: Plural “We show these visualization for different”  these visualizations
P. 7: In “Here our method reaches a mean difference on prediction confidence”  difference in prediction …
P. 7: “This suggest that our method is able”  This suggests that
P. 8: state-of-the-art
P. 8: “has higher mean IoU”  has a higher mean IoU
Whole document: when using “i.e.” add “,” after: i.e.,
References: Some of the references in the list have very little information to be able to find it/proper academic citation, e.g. , Yosinski et al. 2015; Vedaldi and Lenc, 2015:
Jason Yosinski, Jeff Clune, Anh Mai Nguyen, Thomas J. Fuchs, and Hod Lipson. Understanding neural networks through deep visualization. 2015.
A. Vedaldi and K. Lenc. Matconvnet: Convolutional neural networks for matlab. In MM, 2015.
Ref Doersch et al.: What makes paris look like paris?  Paris


NEURAL PROGRAM REPAIR BY JOINTLY LEARNING TO LOCALIZE AND REPAIR
Due to its potential to improve programmer productivity and software quality, automated program repair has been an active topic of research. Newer techniques harness neural networks to learn directly from examples of buggy programs and their fixes. In this work, we consider a recently identified class of bugs called variable-misuse bugs. The state-of-the-art solution for variable misuse enumerates potential fixes for all possible bug locations in a program, before selecting the best prediction. We show that it is beneficial to train a model that jointly and directly localizes and repairs variable-misuse bugs. We present multi-headed pointer networks for this purpose, with one head each for localization and repair. The experimental results show that the joint model significantly outperforms an enumerative solution that uses a pointer based model for repair alone.

This paper presents an LSTM-based model for bug detection and repair of a particular type of bug called VarMisuse, which occurs at a point in a program where the wrong identifier is used. This problem is introduced in the Allamanis et al. paper. The authors of the paper under review demonstrate significant improvements compared to the Allamanis et al. approach on several datasets.
I have concerns with respect to the evaluation, the relation of the paper compared to the state-of-the-art in automatic program repair (APR), and the problem definition with respect to live-variable analysis.
My largest concern about both this paper and the Allamanis et al. paper is how it compares to the state-of-the-art in APR in general. There is a large and growing amount of work in APR as shown in the following papers:
[1] L. Gazzola, D. Micucci, and L. Mariani, “Automatic Software Repair: A Survey,” IEEE Transactions on Software Engineering, pp. 1–1, 2017.
[2] M. Monperrus, “Automatic Software Repair: A Bibliography,” ACM Comput. Surv., vol. 51, no. 1, pp. 17:1–17:24, Jan. 2018.
[3] M. Motwani, S. Sankaranarayanan, R. Just, and Y. Brun, “Do automated program repair techniques repair hard and important bugs?,” Empir Software Eng, pp. 1–47, Nov. 2017.
Although the proposed LSTM-based approach for VarMisuse is interesting, it seems to be quite a small delta compared to the larger APR research space. Furthermore, the above papers on APR are not referenced.
The paper under review mostly uses synthetic bugs. However, they do have a dataset from an anonymous industrial setting that they claim is realistic. In such a setting, I would simply have to trust the blinded reviewers. However, the one industrial software project tells me little about the proposed approach’s effectiveness when applied to a significant number of widely-used software programs like the ones residing in state-of-the-art benchmarks for APR, of which there are at least the following two datasets:
[4] C. L. Goues et al., “The ManyBugs and IntroClass Benchmarks for Automated Repair of C Programs,” IEEE Transactions on Software Engineering, vol. 41, no. 12, pp. 1236–1256, Dec. 2015.
[5] R. Just, D. Jalali, and M. D. Ernst, “Defects4J: A Database of Existing Faults to Enable Controlled Testing Studies for Java Programs,” in Proceedings of the 2014 International Symposium on Software Testing and Analysis, New York, NY, USA, 2014, pp. 437–440.
The above datasets are not used or referenced by the paper under review.
My final concern about the paper is the formulation of live variables. A variable is live at certain program points (e.g., program statements, lines, or tokens as called in this paper). For example, from Figure 1 in the paper under review, at line 5 in (a) and (b), object_name and subject_name are live, not just sources.  In the problem definition, the authors say that "V_def^f \subseteq V denotes the set of all live variables", which does not account for the fact that different variables are alive (or dead) at different points of a program. The authors then say that, for the example in Figure 1, "V_def^f contains all locations in the program where the tokens in V appear (i.e., tokens in the Blue boxes), as well as token sources from line 1”. The explanation of the problem definition when applied to the example does not account for the fact that different variables are alive at different program points. I’m not sure to what extent this error negatively affects the implementation of the proposed model. However, the error could be potentially quite problematic.


Deep learning methods perform well in various tasks. However, the over-fitting problem, which causes the performance to decrease for unknown data, remains. We hence propose a method named MixFeat that directly creates latent spaces in a network that can distinguish classes. MixFeat mixes two feature maps in each latent space in the network and uses unmixed labels for learning. We discuss the difference between a method that mixes only features (MixFeat) and a method that mixes both features and labels (mixup and its family). Mixing features repeatedly is effective in expanding feature diversity, but mixing labels repeatedly makes learning difficult. MixFeat makes it possible to obtain the advantages of repeated mixing by mixing only features. We report improved results obtained using existing network models with MixFeat on CIFAR-10/100 datasets. In addition, we show that MixFeat effectively reduces the over-fitting problem even when the training dataset is small or contains errors. MixFeat is easy to implement and can be added to various network models without additional computational cost in the inference phase.

This paper follows a recent trend to improve generalization by mixing data from training samples, in this case by mixing feature maps from different samples in the latent space. One of the feature maps is added as a kind of perturbation to the other one, so only the label from the main feature map is used as the learning target. MixFeat, the proposed method of adding ‘noise’ from another learning sample is tested on CIFAR-10 and CIFAR-100 with different architectures. The authors claim that the proposed method makes the latent space more discriminative. Multiple experiments show that it helps to avoid over-fitting.
The core idea of mixing the latent spaces of two data samples is interesting and the results seem to indicate that it improves generalization, but I have two main criticisms of this work. First, it is unclear as to why this this approach works (or why it works better than similar methods) and the explanations offered are not satisfactory. The phrase “making features discriminative in the latent space” is used repeatedly, but it is not obvious exactly what is meant by this. Design choices are also not clearly motivated, for example what is the advantage of defining a and b as was done? The second criticism is that comparisons to manifold mixup should have been included.
Approach:
- In “1 Introduction”, the second contribution of presenting “a guideline for judging whether labels should be mixed when mixing features for an individual purpose” is not clearly communicated.
- Figure 1 is a nice idea to illustrate the types of mixed feature distributions, but is not convincing as a toy example. A visualization of how mixed features are placed in the learned latent space for real data would be more informative. The examples showing 0.4A+0.6B and 0.6A+0.4B are confusing - it’s not clear exactly how it relates to the formulation in (1).
- In “2.3 Computation of MixFeat” there is no clear explanation on why the authors chose a and b. Can they just be some other random small values? Is it necessary to have this correlation (cos and sin) between two feature maps we want to mix? Questions like these are not clearly explained. Similar questions can be applied to formula (4) and (6).
+ Explicitly pointing out how backpropagation works for MixFeat in (2) (5) (7) and Figure 2 is helpful.
Experiments:
- The authors mentioned important related works in both “1 Introduction” and “4 Relationship with Previous Work”, but in Table 1, they compared the MixFeat with only standard Mixup. Manifold Mixup would be a  better comparison as it has better performance than standard mixup and is more closely related to MixFeat - MixFeat mixes features in every latent space while Manifold Mixup does in a randomly selected space (and standard mixup only mixes the inputs).
- The method could be described as "adding some noise along samples' latent feature directions". An interesting perspective, and would have been nice to see a comparison of MixFeat vs. perturbing with gaussian noise to see how much the direction towards other examples helps.
+ The experiments to demonstrate the effectiveness of MixFeat for avoiding over-fitting are strong (aside from the missing baseline). The experiments showing robustness to different incorrect label ratios and with different training data size are convincing.
- In Figure 6 center, the x-axis is  or ( for original MixFeat and 1D-MixFeat, and  for Inner-MixFeat), but the authors didn’t make a clear distinction in both the figure caption and “3.3.1 Dimensions and Direction of the Distribution”, having it wrong for Inner-MixFeat with “6.94% ( = 0.02)” which should be “( = 0.02)”.
+ The ablation study motivating the choice of where to apply MixFeat was appreciated.
Related works
+ Clearly presented and covered the relevant literature.
- It would be helpful if the differences between MixFeat and the Mixup family is more clearly stated.


This paper presents a methodology and numerical algorithms for constructing accelerated gradient flows on the space of probability distributions. In particular, we extend the recent variational formulation of accelerated gradient methods in Wibisono et al. (2016)  from vector valued variables to probability distributions. The variational problem is modeled as a mean-field optimal control problem. The maximum principle of optimal control theory is used to derive Hamilton's equations for the optimal gradient flow. The Hamilton's equation are shown to achieve the accelerated form of density transport from any initial probability distribution to a target probability distribution. A quantitative estimate on the asymptotic convergence rate is provided based on a Lyapunov function construction, when the objective functional is displacement convex. Two numerical approximations are presented to implement the Hamilton's equations as a system of N interacting particles. The continuous limit of the Nesterov's algorithm is shown to be a special case with N = 1. The algorithm is illustrated with numerical examples.

This paper derives accelerated gradient flow formula in the space of probability measures from the view of optimal control formalism. The generalization of variational formulation from finite space to the space of probability measures seems new, but the resulting PDE seems to be a known result, which is the Fokker-Planck equation (with some minor modifications) for the 2nd order Langevin dynamic. From this point of view, the resulting algorithm from the derived PDE seems not having much practical advantage over SGHMC (a stochastic version of 2nd order Langevin dynamics).
Actually, I think the derivation of accelerated gradient flow formula from the view of optimal control formalism does not seem necessary. One can get the same formula by deriving it from Wasserstein gradient flows. When considering the functional as relative entropy, one can derive the formula simply from the Fokker-Planck equation of 2nd order Langevin dynamics. As a result, the proposed methods seems to be a new way to derive the Wasserstein gradient flow (or Fokker-Planck equation), which does not make impact the algorithm, e.g., both ways result in the same algorithm.
Besides, I found the writing needs to be improved. There are a lot of background missing, or the descriptions are not clear enough.  For example:
1. Page 2: the divergence operator is not defined, though I think it is a standard concept, but would be better to define it.
2. Page 2: the Wasserstein gradient and Gateaux derivative are not defined, what are the specific meanings of \nabla_\rho F(\rho) and \partial F / \partial \rho?
3. 1st line in Section 2: convex function f of d real variables seems odd, I guess the author means argument of f is d-dimensional variable.
4. Section 2, the authors directly start with the variational problem (3) without introducing the problem. Why do we need to variational problem? It would be hard to follow for some one who does not have such background.
5. Similarly, what is the role of Lyapunov function here in (6)? Why do we need it?
6. Why do you define the Lagrangian L in the form of (10)? What is the relation between (10) and (2)?
7. It is not clear what "The stochastic process (X_t, Y_t) is Gaussian" means in Proposition 1? It might need to be rephrased.
8. Second last line in page 5: I guess \nabla \log(\rho) should be \nabla\log(\rho_t).
For the theory, I think eq.15 only applies when the PDE, e.g. (13), is solved exactly, thus there is not too much practical impact, as it is well known from the Wasserstein gradient theory that the PDE decays exponentially, as stated in the theorem. When considering numerical solutions, I think this results is useless.
For the relation with SGHMC, let's look at eq.16. Actually, the derivative of the log term \nabla \log \rho_t(X_t)) is equivalent to a brownian motion term. This can be seen by considering the Fokker-Planck equation for Brownian motion, which is exactly d \rho_t = \Delta \rho_t. Consequently, instead of using the numerical approximations proposed later, one cane simply replacing this term with a Brownian motion term, which reduces to SGHMC (with some constant multipliers in front).
The authors then shows empirically that the proposed method is better than SGHMC, which I think only comes from the numerical methods.
For the kernel approximation, it makes the particles in the algorithm interactive. This resembles other particle optimization based algorithms such as SVGD, or the latest particle interactive SGLD proposed in [1] or [2[.  I think these methods need to be compared.
[1] Chen et al (2018), A Unified Particle-Optimization Framework for Scalable Bayesian Sampling.
[2] Liu et al (2018), https://arxiv.org/pdf/1807.01750.pdf
To sum up, though the derivation of accelerated gradient flow formula seems interesting, the resulting algorithm does not seem benefit from this derivation. The algorithm seems to be able to derived from a more direct way of using Wasserstein gradient flows, which results in a Wasserstein gradient flow for 2nd order Langevin dynamics, and is thus well known. The experiments are not convincing, and fail to show the advantage of the proposed method. The proposed method needs to be compared with other related methods.


This paper proposes a Pruning in Training (PiT) framework of learning to reduce the parameter size of networks. Different from existing works, our PiT framework employs the sparse penalties to train networks and thus help rank the importance of weights and filters. Our PiT algorithms can directly prune the network without any fine-tuning. The pruned networks can still achieve comparable performance to the original networks. In particular, we introduce the (Group) Lasso-type Penalty (L-P /GL-P), and can alternatively implement them using (Group) Split LBI (S-P / GS-P)-an regularization solution path with corresponding penalties to regularize the networks, and a pruning strategy proposed by Fu et al. (2016b)  is used to help prune the network. We conduct the extensive experiments on MNIST, Cifar-10, and miniImageNet. The results validate the efficacy of our proposed methods. Remarkably, on MNIST dataset, our PiT framework can produce a small network which only has 17.5% parameter size of LeNet-5, and achieves the 98.47% recognition accuracy.

This paper introduces an approach to pruning while training a network. This is interesting and experiments show interesting results in several datasets including ResNet18
Here are a few comments:
 - Pruning or regularization for compression is not new. Alvarez and Wen have used group lasso types as suggested in the paper and some others such as Alvarez and Salzmann (Compression aware training NIPS 2017) and Wen (Coordinating filters ICCV2017) have used low-rank type of while training. How is this different to those? They also do not need any sort of fine tuning and more importantly, they show this can scale to large networks and datasets.
- These last two works I mentioned promote redundancy, similarly to what is suggested in the paper. Would be good to get them cited and compared. Important from those is the training methodology to avoid relevant overheads. How is that happening in the current approach
- While I like the approach, would be nice to see how this scale.  All for methods above (and others related) do work on full imagenet to show performance.  For ResNet, cleaning the network is not really trivial (near the block), is that a limitation?
- Why limiting experiments to small networks and datasets? Time wise, how does this impact the training time?
- Why limiting the experiments to at most 4 layers?
- I am certainly not impressed by results on fully connected layers in MNIST. While the experiment is interesting does not seem to be of value as most networks do not have those layers anymore.
- Main properties of this approach are selecting right filters while training without compromising accuracy or needing fine tuning. While that is of interest, i do not see the difference with other related works (such as those I cited above)
- As there is enough space, I would like to see top-1 results for comprehensive comparison.
- I think tables need better captions for being self-contained. I do not really understand what i see in table 5 for instance.
- Droping 13% of top5 accuracy does not seem negligible, what is the purpose there? Would also be interesting to compare then with any other network with that performance.
- What about flops and forward time? Does this pruning strategy help there?


LEARNING NEURAL PDE SOLVERS WITH CONVER- GENCE GUARANTEES
Partial differential equations (PDEs) are widely used across the physical and computational sciences. Decades of research and engineering went into designing fast iterative solution methods. Existing solvers are general purpose, but may be sub-optimal for specific classes of problems. In contrast to existing hand-crafted solutions, we propose an approach to learn a fast iterative solver tailored to a specific domain. We achieve this goal by learning to modify the updates of an existing solver using a deep neural network. Crucially, our approach is proven to preserve strong correctness and convergence guarantees. After training on a single geometry, our model generalizes to a wide variety of geometries and boundary conditions, and achieves 2-3 times speedup compared to state-of-the-art solvers.

==Summary==
This paper is well-executed and interesting. It does a good job of bridging the gap between distinct bodies of literature, and is very in touch with modern ML ideas.
I like this paper and advocate that it is accepted. However, I expect that it would have higher impact if it appeared in the numerical PDE community. I encourage you to consider this conference paper to be an early version of a more comprehensive piece of work to be released to that community.
My main critique is that the paper needs to do a better job of discussing prior work on data-driven methods for improving PDE solvers.
==Major comments==
* You need to spend considerably more space discussing the related work on using ML to improve PDE solvers. Most readers will be unfamiliar with this. You should explain what they do and how they are qualitatively different than your approach.
* You do a good job 3.3 of motivating for what H is doing. However, you could do a better job of motivating the overall setup of (6). Is this a common formulation? If so, where else is it used?
* I’m surprised that you didn’t impose some sort of symmetry conditions on the convolutions in H, such as that they are invariant to flips of the kernel. This is true, for example, for the linearized Poisson operator.
==Minor comments==
* Valid iterators converge to a valid solution. However, can’t there be multiple candidate solutions? How would you construct a method that would be able to find all possible solutions?
* In (9), why do you randomize the value of k? Wouldn’t you want to learn a different H depending on what computation budget you knew you were going to use downstream when you deploy the solver?
* In future work it may make sense to learn a different H_i for each step i of the iterative solver.
* When introducing iterative solvers, you leave it as an afterthought that b will be enforced by clamping values at the end of each iteration. This seems like a pretty important design decision. Are there alternatives that guarantee that u satisfies b always, rather than updating u in such a way that it violates G and then clamping it back? Along these lines, it might be useful to pose (2) with additional terms in the linear system to reflect G.


The remarkable success of machine learning, especially deep learning, has produced a variety of cloud-based services for mobile users. Such services require an end user to send data to the service provider, which presents a serious challenge to end-user privacy. To address this concern, prior works either add noise to the data or send features extracted from the raw data. They struggle to balance between the utility and privacy because added noise reduces utility and raw data can be reconstructed from extracted features. This work represents a methodical departure from prior works: we balance between a measure of privacy and another of utility by leveraging adversarial learning to find a sweeter tradeoff. We design an encoder that optimizes against the reconstruction error (a measure of privacy), adversarially by a Decoder, and the inference accuracy (a measure of utility) by a Classifier. The result is RAN, a novel deep model with a new training algorithm that automatically extracts features for classification that are both private and useful. It turns out that adversarially forcing the extracted features to only conveys the intended information required by classification leads to an implicit regularization leading to better classification accuracy than the original model which completely ignores privacy. Thus, we achieve better privacy with better utility, a surprising possibility in machine learning! We conducted extensive experiments on five popular datasets over four training schemes, and demonstrate the superiority of RAN compared with existing alternatives.

Privacy concerns arise when data is shared with third parties, a common occurrence. This paper proposes a privacy-preserving classification framework that consists of an encoder that extracts features from data, a classifier that performs the actual classification, and a decoder that tries to reconstruct the original data. In a mobile computing setting, the encoder is deployed at the client side and the classification is performed on the server side which accesses only the output features of the encoder. The adversarial training process guarantees good accuracy of the classifier while there is no decoder being able to reconstruct the original input sample accurately. Experimental results are provided to confirm the usefulness of the algorithm.
The problem of privacy-preserving learning is an important topic and the paper proposes an interesting framework for that. However, I think it needs to provide more solid evaluations of the proposed algorithm, and presentation also need to be improved a bit.
Detailed comments:
I don’t see a significant difference between RAN and DNN in Figure 5. Maybe more explanation or better visualization would help.
The decoder used to measure privacy is very important. Can you provide more detail about the decoders used in all the four cases? If possible, evaluating the privacy with different decoders may provide a stronger evidence for the proposed method.
It seems that DNN(resized) is a generalization of DNN. If so, by changing the magnitude of noise and projection dimensions for PCA should give a DNN(resized) result (in Figure 3) that is close to DNN. If the two NNs used in DNN and DNN(resized) are different, I believe it’s still possible to apply the algorithm in DNN(resized) to the NN used in DNN, and get a full trace in the figure as noise and projection changes, which would lead to more fair comparison.
The abstract mentioned that the proposed algorithm works as an “implicit regularization leading to better classification accuracy than the original model which completely ignores privacy”. But I don’t see clearly from the experimental results how the accuracy compares to a non-private classifier.
Section 2.2 mentioned how different kind of layers would help with the encoder’s utility and privacy. It would be better to back up the argument with some experiments.
I think it needs to be made clearer how reconstruction error works as a measure of privacy. For example, an image which is totally unreadable for human eye might still leak sensitive information when fed into a machine learning model.
In term of reference, it’s better to cite more articles with different kind of privacy attacks for how raw data can cause privacy risks. For the “Noisy Data” method, it’s better to cite more articles on differential privacy and local differential privacy.
Some figures, like Figure 3 and 4, are hard to read. The author may consider making the figures larger (maybe with a 2 by 2 layout), adjusting the position of the legend & scale of x-axis for Figure 3, and using markers with different colors for Figure 4.


Personalized probabilistic forecasts of time to event (such as mortality) can be crucial in decision making, especially in the clinical setting. Inspired by ideas from the meteorology literature, we approach this problem through the paradigm of maximizing sharpness of prediction distributions, subject to calibration. In regression problems, it has been shown that optimizing the continuous ranked probability score (CRPS) instead of maximum likelihood leads to sharper prediction distributions while maintaining calibration. We introduce the Survival-CRPS, a generalization of the CRPS to the time to event setting, and present right-censored and interval-censored variants. To holistically evaluate the quality of predicted distributions over time to event, we present the scale agnostic Survival-AUPRC evaluation metric, an analog to area under the precision-recall curve. We apply these ideas by building a recurrent neural network for mortality prediction, using an Electronic Health Record dataset covering millions of patients. We demonstrate significant benefits in models trained by the Survival-CRPS objective instead of maximum likelihood.

The authors introduce an extension of Continuous Ranked Probability Scores (CRPS) to the time-to-event setting termed Survival-CRPS for both right censored and interval-censored event data. Further, the authors introduce a scale agnostic Survival-AUPRC evaluation metric that is analogous to the precision-recall curve used in classification and information retrieval systems/models.
The claim that that the proposed approach constitutes the first time a scoring rule other than maximum likelihood seems too strong, unnecessary and irrelevant to the value of the presented work.
It is not clear how did the authors handle the irregularity (in time) of EHR encounters in the context of an RNN specification. Also, if the RNN specification considered is similar to Martinsson, 2016, why this wasn't considered as a competing model in the experiments?
In Table 1 , it is not clear what the error bars are also they seem too small.
The proposed approach addresses important questions in time-to-event modeling, namely, calibration and interval censoring. Although the connection with CRPS is interesting (first of the two equations in page 3), it is quite similar to an accelerated failure time formulation, which for a log-normal specification is standard and popular due to similar reasons to those highlighted by the authors, but not mentioned in the related work. The interval censoring is also interesting, though straightforward and perhaps not as relevant in more general time-to-event settings where events other than age are considered.
The Survival-AUPRC is not sufficiently motivated. Without motivation or an intuition of why it should be used/preferred, it seems disconnected from the rest of the paper and its contributions.
Without a more comprehensive evaluation that includes additional datasets and competing models (described in the Related Work Section) it is difficult to assess the value of the proposed approach.


ADAPTIVE ESTIMATORS SHOW INFORMATION COM- PRESSION IN DEEP NEURAL NETWORKS
To improve how neural networks function it is crucial to understand their learning process. The information bottleneck theory of deep learning proposes that neural networks achieve good generalization by compressing their representations to disregard information that is not relevant to the task. However, empirical evidence for this theory is conflicting, as compression was only observed when networks used saturating activation functions. In contrast, networks with non-saturating activation functions achieved comparable levels of task performance but did not show compression. In this paper we developed more robust mutual information estimation techniques, that adapt to hidden activity of neural networks and produce more sensitive measurements of activations from all functions, especially unbounded functions. Using these adaptive estimation techniques, we explored compression in networks with a range of different activation functions. With two improved methods of estimation, firstly, we show that saturation of the activation function is not required for compression, and the amount of compression varies between different activation functions. We also find that there is a large amount of variation in compression between different network initializations. Secondary, we see that L2 regularization leads to significantly increased compression, while preventing overfitting. Finally, we show that only compression of the last layer is positively correlated with generalization.

This paper has 3 principal contributions: it proposes a different way of measuring mutual information in a neural network, proposes a compression score to compare different models, then empirically analyses different activation functions and L2 weights.
This work seems like a welcome addition to the IB thread. To me the most interesting result is simply that activation functions aren't simply about gradient flow, and that they may each have properties that are more or less desirable depending on the domain they might be used on. The authors are careful in the wording of their conclusions, I think with reason; while these results are useful in that there seem to be consistently different behaviors coming from different hyperparameters, information planes show a relatively qualitative part of the picture.
Quantitatively, the proposed compression score is interesting, but as the authors say, simplistic. It seems to me that we care more about the converged models than the whole training trajectory; how does this score evolve with time?
I think an important part of discussion that lacks in this paper is a more in-depth take as to how these findings relate to the Zhang et al [1] memorization vs generalization paper and its follow ups. There seem to be many links to be drawn.
This work is overall a good contribution, but I'll have to agree with the authors' conclusion that more principled analysis methods are required to have a solid grasp of the training dynamics of DNNs. The writing of the paper is good, but the writing of the captions could be improved. (the hard page limit of ICLR is 10 pages and your paper has a lot of captions, so I think investing into a bit more text would be good)
Comments:
- It might be worth to re-explain what the information plane plots are in a figure caption, not just in the text (the text also doesn't really explain that each point is a moment in training, and each thread a different layer, this paper should be readable by someone who has never seen these plots before).
- It's not clear what is going on in figure 5, I can guess but, again, this paper should be readable by anyone in the field. You mention different initializations, but which exactly? What makes you say that 5c has no compression but that 5a does compression first? It should be explained explicitly.
- I believe what you say about Figure 8, but the plots are so similar that it is hard to compare them visually. Maybe a different kind of superposition into a single plot would better illustrate the compression effect of L2?
- Typo in the x axis caption of figures 9.
- Figure 9a is not readable in greyscale (or by a colorblind person), consider using a different symbol for the softmax scatter (and adding this symbol to the legend).
- The first Schmidhuber citation of the paper seems a bit out of place. I think he himself would say that deep learning has been going on for much longer than since 2015. (in fact I think you could just remove the entire first paragraph, it is just unnecessary boilerplate)
- Why should there be a direct correlation between compression and generalization? For example, it is known that training DNNs with soft targets improves test accuracy in classification, or even forcing softness in both targets and representations [2] also improves test accuracy.
- I'm still personally not sold on binning as a strategy to evaluate MI. Did you perform experiments that show that the observed difference is consistent if more computation is done to approximate MI, and not just an artefact of max-entropy binning?
[1] Zhang et al (2016) https://arxiv.org/abs/1611.03530
[2] Verma et al (2018) https://arxiv.org/abs/1806.05236


We propose to improve trust region policy search with normalizing flows policy. We illustrate that when the trust region is constructed by KL divergence constraint, normalizing flows policy can generate samples far from the 'center' of the previous policy iterate, which potentially enables better exploration and helps avoid bad local optima. We show that normalizing flows policy significantly improves upon factorized Gaussian policy baseline, with both TRPO and ACKTR, especially on tasks with complex dynamics such as Humanoid.

The papers proposed to use normalizing flow policies instead of Gaussian policies to improve exploration and achieve better sample complexity in practice. While I believe this idea has not specifically been tried in previous literature and the vague intuition that NF leads to more exploration that helps learning a better policy, the novelty of combining these two seems limited, and the paper does not seem to provide enough justification to using NF policies instead of alternative policy distributions both in theory and in the experiments.
1. About Section 4.2. I believe that the normalizing flow in question would transform the volume of a Gaussian? So there would exist some parameter setting for a flow model that also shrinks volume, thereby resulting in lower variance policies? The arguments would thereby depend heavily on the specific architecture and initialization of the flow model, which is not discussed in detail.
Also, why is finding a high variance policy better in terms of the trust region argument? Isn't the whole point of using trust region that the new policy should be closer to old policy to prevent performance degradation? I also think that a fair comparison would be compare KL between normalizing flow policies, instead of KL between NF and Gaussian.
2. The TRPO experiments seem wrong -- at least the results don't match what is reported in the ACKTR paper for Reacher and InverseDoublePendulum envs -- there the TRPO policy at least learns something. Also TRPO in general does not perform as bad as it may seem, see "Deep RL that matters" paper by Henderson et al. Maybe this is because of using OpenAI baselines code which seems to have worse TRPO performance.
There is also no experiments on ACKTR on the small Mujoco tasks (even in the Appendix), which seems to be a rather big oversight given the authors have already done even harder tasks for ACKTR + NF.
Moreover I think a fair comparison is to use almost the same architecture for implicit and gaussian, where the only difference is where you sample the noise. For Gaussian with flows, you can first use an MLP to produce deterministic outputs and then use flow to generate the mean actions. Otherwise it is impossible to say whether the architecture or the implicit distribution contributes more to the success.
One could also use truncated Gaussian distributions / Beta distributions / Gaussian + tanh, since Mujoco actions beyond (-1, 1) is treated as -1 or 1, so Gaussian should already be bad. It is unclear whether NF is able to outperform these settings.
Minor points:
- Fix citations. Please use \citep throughout.
- Is Equation (6) correct? Seems like \Sigma_i should be the inverse of g_i(\epsilon)? Also this is the "change of variables formula" not "chain rule".
- Why is normalizing flow not part of the background?
- Add legends in Figure (1)
- Figure 2(c), I believe with max entropy you could already obtain diverse ant trajectories?
- I believe in the context of generative models, "implicit" typically means the case where likelihood is not tractable? Here the likelihood is perfectly tractable.


Over the last few years, the phenomenon of adversarial examples -maliciously constructed inputs that fool trained machine learning models -has captured the attention of the research community, especially when the adversary is restricted to making small modifications of a correctly handled input. At the same time, less surprisingly, image classifiers lack human-level performance on randomly corrupted images, such as images with additive Gaussian noise. In this work, we show that these are two manifestations of the same underlying phenomenon. We establish this connection in several ways. First, we find that adversarial examples exist at the same distance scales we would expect from a linear model with the same performance on corrupted images. Next, we show that Gaussian data augmentation during training improves robustness to small adversarial perturbations and that adversarial training improves robustness to several types of image corruptions. Finally, we present a model-independent upper bound on the distance from a corrupted image to its nearest error given test performance and show that in practice we already come close to achieving the bound, so that improving robustness further for the corrupted image distribution requires significantly reducing test error. All of this suggests that improving adversarial robustness should go hand in hand with improving performance in the presence of more general and realistic image corruptions. This yields a computationally tractable evaluation metric for defenses to consider: test error in noisy image distributions.

This paper propose an alternative view for adversarial examples in high dimension spaces by considering the "error rate" in a Gaussian distribution centered at each test point. However, as mentioned in the related work, adversarial examples through the lens of isoperimetric inequality is not new to this paper; the implication of adversarial sensitivity by error rate in the test-sample-centered-Gaussian in general non-linear case is rather weak; and the empirical results does not show advantage over simple adversarial training against lp constrained adversarial attacks.
Here are some more detailed comments and feedbacks:
The clarity could be improved by making clear use of notations and define some key terms explicitly:
1. For example, the error rate sometimes refer to the test error, sometimes refer to the error rate under a special distribution centered at a particular test example.
2. Similarly, the distribution q sometimes refer to the original (unknown) input data distribution, but the same notation is also used to refer to this Gaussian distribution centered on each test example. Although the paper says that "q need not be restricted to the distribution from which the training set was sampled", it could potentially confuse the reader less if there is a symbol for the "usual" test error and a different one for this test-error-under-Gaussian-centered-at-a-particular-test-example.
3. It would be good if the paper could make a formal definition of the problem being studied and explicitly specify the assumptions on the existence of a deterministic target function (concept) and explicitly define the error set E.
4. It seems to assume the input distribution is continuous everywhere but not stated. If for example, the original data is supported on disconnected manifolds separated with low density or even zero-density margins, then the Gaussian distribution centered on test example argument will need to be modified to talk about the intersection of the Gaussian with the data manifold instead. If the paper does decide to make this kind of assumption, some empirical study on the data to verify the fidelity of the assumptions would be great.
5. The paper does not mention how measurement against the error set E. Under the original data distribution, it is natural to measure the error rate with the provided training or test data with labels. However, under each newly formed Gaussian distribution centered at each test point, the labels for the newly sampled examples from this Gaussian is unknown, and since there is no "ground truth classifier" for MNIST or CIFAR10 available, it seems impossible to "calculate" the true label for those samples, which are needed to calculate the error rate. It is not very clear from the paper how this issue is solved. I'm guess it uses the label from the Gaussian center for all the samples from the Gaussian. While this might be reasonable assumption for Gaussian with tiny variances, it is less clear how reasonable it is for the large variance Gaussian distributions considered in the paper. If this assumption is made, please state it explicitly and empirically or theoretically study how reasonable this assumption is in the regime of variances considered in this paper. If my guessing is wrong, please also explicitly what approach is used to get around this issue.
The followings are some feedbacks on the contents and ideas of the paper:
6. I think one sentence in the text summarize a large part of the paper very well: "to measure adversarial robustness is to ask whether or not there are any errors in the linf ball, ... and to measure test error in noise is to measure the volume of the error set in the defined noise distribution". However, this looks like a rather roundabout approach to attack another problem (measuring volume of an unknown set in a very high dimension space) in order to solve the original problem, while the implication is rather weak (a precise implication can only be obtained for linear separating hyper-plane, while for non-linear classifiers it is much less clear).
Note while exact adversarial robustness is NP-hard, volume estimation in high dimension is not easy (if not harder). For cifar-10, the inputs are in dimension 32x32x3 = 3072, the 1,000 samples used in the paper to estimate the volume of a set in this high dimension seem to be quite inaccurate. I would appreciate if variances could be reported in those studies to show the confidence of the estimations. For imagenets, the inputs are in even larger spaces.
Given the difficulty (in terms of sample complexity) to estimate the "error-in-noise", it might not be very surprising that the noise augmentation does not show advantages to lp constrained adversarial attacks (comparing to adversarial training).
7. In the conclusion, the paper states "we proved a fundamental relationship between generalization in noisy image distributions and the existence of small adversarial perturbations". I believe a formal proof is only given to the case of existence of small adversarial perturbations to examples to the "noisy examples" from the Gaussian distribution, and in this case, it is a rather direct corollary from the Gaussian Isoperimetric Inequality. For the more "practical case" (in the sense that is more related to the usual notion of adversarial examples) of existence of small adversarial examples, it seems only the case of the linear classifier is formally discussed.
Moreover, I'm a little bit worried some important pieces might be lost and create potentially misleading or seemly strong conclusion. Maybe it would be helpful if a concrete example could be given in the paper that shows the full path from the error-in-noise to existence of adversarial example, by showing all the constants involved. I'm a bit confused here because (in order for the isoperimetric inequality to have favorable bounds?) the Gaussian distributions used in error-in-noise seem to have rather large variance. As mentioned in the paper, the majority of the mass in the Gaussian distribution considered will be in a thin sphere of radius sigma * sqrt(n) centered at the test example x. If sigma = 0.1 and dimension n = 3072 (cifar-10), then the radius is around 5.5 (in l2 distance) which is probably quite far from the test point x (is it?). It is then less clear how "a large majority of this thin sphere far away from x is epsilon close to the error set" could tightly imply properties of adversarial robustness of x itself in its close vicinity. Maybe a specific example with all the numerical constants spelled out would help illustrate this.
In summary, I think this paper takes an interesting but roundabout perspective to adversarial robustness, and the implication is weak in the non-linear case. (Potentially because of the weak implication), the suggested approach for defenses by augmenting with noises does not show advantage over adversarial training.


We formulate an information-based optimization problem for supervised classification. For invertible neural networks, the control of these information terms is passed down to the latent features and parameter matrix in the last fully connected layer, given that mutual information is invariant under invertible map. We propose an objective function and prove that it solves the optimization problem. Our framework allows us to learn latent features in a more interpretable form while improving the classification performance. We perform extensive quantitative and qualitative experiments in comparison with the existing state-of-the-art classification models.

In this paper, the authors propose to train a model from the point of view of maximizing mutual information between the predictions and the true outputs, with a regularization term that minimizes irrelevant information while learning. They show that the objective can be minimized by looking to make the final layer vectors be as uncorrelated as possible to the final layer representations, and simplify the same by applying Holder’s inequality to make the optimization tractable. They also apply an L1 penalty on the final layer. Experiments on CIFAR and MNIST show that using their regularizer to train DNN models yield  gains in performance. The presence of the L1 penalty also makes the results more interpretable (to the extend possible by looking at a subset of features in the last layer of a DNN).
COMMENTS:
- Meta Point: To really see that the regularization framework you’re proposing is good, why not just pick a simple, feedforward model or convNet, see the performance and then compare it with the regularizer you’re proposing? That will help hit the point home.
- Page 1: before jumping to equations (1) and (2), please formally define Mutual Information. The actual definition is much later in the text, but it’s better to define first.
- Beyond  referring the user to section 3 on Page 1, please also mention a couple of key references in the appropriate locations.
- Page 3 paragraph 2: “Mutual information is bounded … correct them” : Can you provide some formulas for this and make this concrete? Or perhaps provide some references? This line is vague.
 - prop 2.1 and 2.2: can you define what you mean by “empirical version”? Again, it’s probably good to have these terms crisply defined before using them.
- eqn (6) is interesting. Holder’s inequality gives you the product terms. Then you can also apply the AM-GM inequality, and get a sum. So then at the end of it all, you’re left with the standard elastic net penalty and not the product form. In that case, aren’t we back to just the usual regularization strategy? And in which case, should I interpret the results you have in sec 4 as “using L1 penalties with L2 is good” ?
- To the point above, I guess one difference after the AM-GM step is that you will not have a squared L2 norm, but just L2. This is reminiscent of linear models where they use L2 loss instead of squared L2 loss. But on the penalty, squaring just adds smoothness. Can you comment on this?
- sec 4.2.1: I don’t see how fig (2) (L) is “roughly Gaussian”. Can you explain? Maybe plot the histogram? Also for fig (2, R): the coefficients are approximately sparse. It’s not sparse as you claim since there are almost no zeros in the coefficients.
- I don’t get the point of sec 4.3: How does this claim not apply to all deep learning models, regardless of the penalizations you propose?
edit:
I have read the responses and the other reviews. The authors have addressed the few major points I had. I still think there are a few gaps that need to be addressed (as pointed by the other reviewers)


Deep neural networks have achieved outstanding performance in many real-world applications with the expense of huge computational resources. The DenseNet, one of the recently proposed neural network architecture, has achieved the state-of-the-art performance in many visual tasks. However, it has great redundancy due to the dense connections of the internal structure, which leads to high computational costs in training such dense networks. To address this issue, we design a reinforcement learning framework to search for efficient DenseNet architectures with layer-wise pruning (LWP) for different tasks, while retaining the original advantages of DenseNet, such as feature reuse, short paths, etc. In this framework, an agent evaluates the importance of each connection between any two block layers, and prunes the redundant connections. In addition, a novel reward-shaping trick is introduced to make DenseNet reach a better trade-off between accuracy and float point operations (FLOPs). Our experiments show that DenseNet with LWP is more compact and efficient than existing alternatives.

The paper introduces RL based approach to prune layers in a DenseNet. This work extends BlockDrop to DenseNet architecture making the controller independent form the input image. The approach is evaluated on CIFAR10 and CIFAR100 datasets as well as on ImageNet showing promising results.
In order to improve the paper, the authors could take into consideration the following points:
1. Given the similarity of the approach with BlockDrop, I would suggest to discuss it in the introduction section clearly stating the similarities and the differences with the proposed approach.
2. BlockDrop seems to introduce a general framework of policy network to prune neural networks. However, the authors claim that BlockDrop "can only be applied to ResNets or its variants". Could the authors comment on this?
3. In the abstract, the authors claim: "Our experiments show that DenseNet with LWP is more compact and efficient than existing alternatives". It is hard to asses if the statement is correct given the evidence presented in the experimental section. It is not clear if the method is more efficient and compact than others, e. g.  CondenseNet.
4. In the experimental section, addressing the following questions would make the section stronger: What is more important FLOPs or number of parameters? What is the accuracy drop we should allow to pay for reduction in number of parameters or FLOPs?
5. For the evaluation, I would suggest to show that the learned policy is better than a random one: e. g. not using the controller to define policy (in line 20 of the algorithm) and using a random random policy instead.
6. In Table 1, some entries for DenseNet LWP are missing. Is the network converging for this setups?
7. \sigma is not explained in section 3.3. What is the intuition behind this hyper parameter?
8. I'd suggest moving related work section to background section and expanding it a bit.
9. In the introduction: "... it achieved state-of-the-art results across several highly competitive datasets". Please add citations accordingly.
Additional comments:
1. It might be interesting to compare the method introduced in the paper to a scenario where the controller is conditioned on an input image and adaptively selects the connections/layers in DenseNet at inference time.
2. It might be interesting to report the number of connections in Table 1 for all the models.
Overall, I liked the ideas presented in the paper. However, I think that the high degree of overlap with BlockDrop should be addressed by clearly stating the differences in the introduction section. Moreover, I encourage the authors to include missing results in Table 1 and run a comparison to random policy. In the current version of the manuscript, it is hard to compare among different methods, thus, finding a metric or a visualization that would clearly outline the "efficiency and compactness" of the method would make the paper stronger.