Machine learning (ML) research has investigated prototypes: examples that are representative of the behavior to be learned. We systematically evaluate five methods for identifying prototypes, both ones previously introduced as well as new ones we propose, finding all of them to provide meaningful but different interpretations. Through a human study, we confirm that all five metrics are well matched to human intuition. Examining cases where the metrics disagree offers an informative perspective on the properties of data and algorithms used in learning, with implications for data-corpus construction, efficiency, adversarial robustness, interpretability, and other ML aspects. In particular, we confirm that the "train on hard" curriculum approach can improve accuracy on many datasets and tasks, but that it is strictly worse when there are many mislabeled or ambiguous examples.

Summary: The paper proposes methods for identifying prototypes. Unfortunately, a formal definition of a prototype is lacking, and the authors instead present a set of heuristics for sorting data points that purport to measure 'prototypicality', although different heuristics have different (and possibly conflicting) notions of what this means. The experiments are not very convincing, and often present results that are either inconclusive or negative, i.e. seem to demonstrate that prototypes are not very useful.
Pros:
- The notion of prototypes is used in various papers, but a formal definition is lacking, and the usefulness of prototypes is not demonstrated. The fact that this paper sets out to do both is laudable, although the paper needs work before it can be accepted for publications.
Detailed comments / cons:
*Defining prototypes:
  - The authors list desirable properties before defining (even informally) what a prototype is, and what its purposes are. Taking the first property as an example, is it reasonable to expect a metric for prototypes to be useful for image classification AND image generation? The answer completely depends on what one expects from a prototype, what its purpose is, etc.
  - The second property seems to indicate that prototypes are model-independent, i.e. two models trained on the same dataset will have the same prototypes. This is confusing as the metrics proposed are clearly model-dependent (e.g. adv completely depends on the trained model's decision boundary, conf obviously depends on the model providing the confidence score)
- The third and fourth property are poorly defined. Human intuition presupposes that humans agree on what a prototype means. Using 'modes of prototypical examples' in trying to define a metric for prototypes is circular, as a mode of prototypical example depends on a working notion of prototypical examples.
- The last property is completely dependent on which models are trained, and how they are trained. If a model has high label complexity, maybe it does not achieve high accuracy even when trained on high quality prototypes. In any case, this property is at odds with the first two properties.
In sum: it's not clear what prototypes are, so it becomes hard to judge if the list of desiderata is reasonable. The list is in any case ill-defined, and contains contradictions.
* Metrics for prototypicality
- The second paragraph in this section is unnecessary
- All of the metrics proposed are heuristics with little to no justification. Specific comments below.
- Adversarial robustness is a property of a trained model, not of prototypical examples, unless prototypes are supposed to be model dependent (contra property 1). In any case, it is not clear why examples that are robust to adversarial noise are good 'prototypes'.  Using facial recognition as an example, a 'mean face' may be very robust to adversarial noise but not prototypical at all under common definitions. A face with a particular type of facial hair (e.g. nose hair) may be very representative of a class of faces (i.e. a prototype), but very susceptible to adversarial noise. In fact, any examples in the boundary of the decision function will be more susceptible to adversaries, but that does not make them 'less prototypical'.
- Holdout retraining is again completely model dependent. Why should we expect a model to treat a prototype the same regardless of whether or not it is trained on it? This basically means that we expect the model to always be accurate on prototypes.
- Ensemble agreement proposes a notion of prototypes that is based on prediction 'hardness'. It is clear that such a notion depends completely on which models are being considered, which features are being used, and etc, much more than on notions of prototypicality inherent in the data. The same criticism applies to model confidence.
- Privacy preserving training assumes prototypicality has to do with the model being able to learn with some robustness to noise (related to Adversarial Robustness, but different). This assumes a definition of prototypes that is not congruent with the other metrics.
In sum: the proposed metrics are basically heuristics with little justification, and different metrics assume different notions of what a prototype is.
* Evaluation
- Section 3.1 claims that the metrics are strongly correlated, but that is not true for MNIST or CIFAR, and is somewhat true for fashion-mnist. In any case, since the metrics are so model-dependent, it is not clear if these results would hold if other models were used.
- Section 3.2 - The question asked of turkers in the study is too vague, and borderline irrelevant for the task at hand - what does the 'best image' of an airplane mean, and how does this translate to it being a prototype? All that the study demonstrates is that the proposed metrics score malformed images with low score. The results in Table 1 are very spread out, and seem to indicate a low agreement between the metrics and human evaluation - although Table 1 is almost irrelevant given the question that was asked of users.
- The results in Section 4 are very discouraging: sometimes it is better to train on most prototypical examples according to the metrics, sometimes it is worse, sometimes it's better to take examples in the middle. That is, prototypes don't seem to help at all. 'Prototype percentile' is uncorrelated with robustness for MNIST in Appendix E, while being correlated for other datasets. It is clear why this would be the case for metrics such as confidence, but in general models trained on less examples are less robust than models trained on the whole dataset (again, as expected). As a whole, the results do not provide any help for a user who wants to produce a more robust model, other than 'ignore prototypes and use the whole dataset'.


PEERNETS: EXPLOITING PEER WISDOM AGAINST ADVERSARIAL ATTACKS
Deep learning systems have become ubiquitous in many aspects of our lives. Unfortunately, it has been shown that such systems are vulnerable to adversarial attacks, making them prone to potential unlawful uses. Designing deep neural networks that are robust to adversarial attacks is a fundamental step in making such systems safer and deployable in a broader variety of applications (e.g. autonomous driving), but more importantly is a necessary step to design novel and more advanced architectures built on new computational paradigms rather than marginally building on the existing ones. In this paper we introduce PeerNets, a novel family of convolutional networks alternating classical Euclidean convolutions with graph convolutions to harness information from a graph of peer samples. This results in a form of non-local forward propagation in the model, where latent features are conditioned on the global structure induced by the graph, that is up to 3× more robust to a variety of white-and black-box adversarial attacks compared to conventional architectures with almost no drop in accuracy.

The paper presents a an interesting novel approach to train neural networks with so called peer regularization which aims to provide robustness to adversarial attacks. The idea is to add a graph neural network to a spatial CNN. A graph is defined over similar training samples which are found using a Monte Carlo approximation.
The regularization using graphs reminds me of recent work at ICML on semi-supervised learning (Kamnitsas et al. (2018) Semi-supervised learning via compact latent space clustering) which is using a graph to approximate cluster density which acts as a regularizer for training on labelled data.
The main problem I see with these approaches is that they rely on sufficiently large batch sizes which could be (currently) problematic for many real-world applications. Memory and computation limitations are mentioned, but not sufficently discussed. It would be good to add further details on practical limitations.
Experiments are limited to benchmark data using MNIST, CIFAR-10, CIFAR-100. Comprehensive evaluation has been carried out with insightful experiments and good comparison to state-of-the-art. Both white- and black-box adversarial attacks are explored with promising results for the proposed approach.
However, it is difficult to draw conclusions for real-world problems of larger scale. The authors state that proposed framework can be added to any baseline model, but miss to clearly mention the limitations. It is stated that future work will aim at scaling PeerNets to benchmarks like ImageNet, but it is unclear how this could be done. Is there any hope this could be applied to problems like 3D imaging data or videos?


Word inputs tend to be represented as single continuous vectors in deep neural networks. It is left to the subsequent layers of the network to extract relevant aspects of a word's meaning based on the context in which it appears. In this paper, we investigate whether word representations can be improved by explicitly incorporating the idea of latent roles. That is, we propose a role interaction layer (RIL) that consists of context-dependent (latent) role assignments and rolespecific transformations. We evaluate the RIL on machine translation using two language pairs (En-De and En-Fi) and three datasets of varying size. We find that the proposed mechanism improves translation quality over strong baselines with limited amounts of data, but that the improvement diminishes as the size of data grows, indicating that powerful neural MT systems are capable of implicitly modeling role-word interaction by themselves. Our qualitative analysis reveals that the RIL extracts meaningful context-dependent roles and that it allows us to inspect more deeply the internal mechanisms of state-of-the-art neural machine translation systems.

[Summary]
This paper proposes “a role interaction layer” (briefly, RIL) that consists of context-dependent (latent) role assignments and role-specific transformations: Given an RIL layer, different dimensions of an embedding vector are “interacted” based on Eqn. (5), Eqn. (7), etc. The authors work on IWSLT De->En and WMT En->De, En->Fi to verify their proposed algorithm with case study included.
[Pros]
(+) I think the idea/thought of using a “role interaction layer” is interesting.  The case study in Section 5.3 demonstrates different “roles”. Also, different RIL architectures are designed.
(+) The paper is easy to follow.
[Cons & Details]
(1) As stated in the abstract, “…, but that the improvement diminishes as the size of data grows, indicating that powerful neural MT systems are capable of implicitly modeling role-word interaction by themselves…” (1) The main concern is that, considering RIL does not obtain significant gain on large datasets, then we cannot say that the proposed algorithm is better than the baseline. (2) Why the NMT systems trained on large dataset can “implicitly modeling role-word interaction”, while small dataset cannot? Any analysis?
(2) For the “matched baseline”, page 5, you increase the dimensionality of the models. But an RIL is an additional layer, which makes the network deeper. Therefore, a baseline with an additional layer should be implemented.


ALISTA: ANALYTIC WEIGHTS ARE AS GOOD AS LEARNED WEIGHTS IN LISTA
Deep neural networks based on unfolding an iterative algorithm, for example, LISTA (learned iterative shrinkage thresholding algorithm), have been an empirical success for sparse signal recovery. The weights of these neural networks are currently determined by data-driven "black-box" training. In this work, we propose Analytic LISTA (ALISTA), where the weight matrix in LISTA is computed as the solution to a data-free optimization problem, leaving only the stepsize and threshold parameters to data-driven learning. This significantly simplifies the training. Specifically, the data-free optimization problem is based on coherence minimization. We show our ALISTA retains the optimal linear convergence proved in (Chen et al., 2018)  and has a performance comparable to LISTA. Furthermore, we extend ALISTA to convolutional linear operators, again determined in a data-free manner. We also propose a feed-forward framework that combines the data-free optimization and ALISTA networks from end to end, one that can be jointly trained to gain robustness to small perturbations in the encoding model.

The papers studies neural network-based sparse signal recovery, and derives many new theoretical insights into the classical LISTA model. The authors proposed Analytic LISTA (ALISTA), where the weight matrix in LISTA is pre-computed with a data-free coherence minimization, followed by a separate data-driven learning step for merely (a very small number of) step-size and threshold parameters. Their theory is extensible to convolutional cases. The two-stage decomposed pipeline was shown to keep the optimal linear convergence proved in (Chen et al., 2018). Experiments observe that ALISTA has almost no performance loss compared to the much heavier parameterized LISTA, in contrast to the common wisdom that (brutal-force) “end-to-end” always outperforms stage-wise training. Their contributions thus manifest in both novel theory results, and the practical impacts of simplifying/accelerating LISTA training.  Besides, they also proposed an interesting new strategy called Robust ALISTA to overcome the small perturbations on the encoding basis, which also benefits from this decomposed problems structure.
The proofs and conclusions are mathematically correct to my best knowledge. I personally worked on similar sparse unfolding problems before so this work looks particularly novel and interesting to me. My intuition then was that, it should not be really necessary to use heavily parameterized networks to approximate a simple linear sparse coding form (LISTA idea). Similar accelerations could have been achieved with line search for something similar to steepest descent (also computational expensive, but need learn step-sizes only, and agnostic to input distribution). Correspondingly, there should exist a more elegant network solution with very light learnable weights. This work perfectly coincides with the intuition, providing very solid guidance on how a LISTA model could be built right. Given in recent three years, many application works rely on unfold-truncating techniques (compressive sensing, reconstruction, super resolution, image restoration, clustering…), I envision this paper to generate important impacts for practitioners pursuing those ideas.
Additionally, I like Theorem 3 in Section 3.1, on the provable efficient approximation of general convolution using circular convolution. It could be useful for many other problems such as filter response matching.
I therefore hold a very positive attitude towards this paper and support for its acceptance. Some questions I would like the authors to clarify & improve in revision:
1.	Eqn (7) assumes noise-free case. The author stated “The zero-noise assumption is for simplicity of the proofs.” Could the authors elaborate which part of current theory/proof will fail in noisy case? If so, can it be overcome (even by less “simpler” way)? How about convolutional case, the same? Could the authors at least provide some empirical results for ALISTA’s performance under noise?
2.	Section 5.3. It is unclear to me why Robust ALISTA has to work better than the data augmented ALISTA. Is it potentially because that in the data augmentation baseline, the training data volume is much amplified, and one ALISTA model might become underfitting? It would be interesting to create a larger-capacity ALISTA model (e.g., by increasing unfolded layer numbers), train it on the augmented data, and see if it can compare more favorably against Robust ALISTA?
3.	The writeup is overall very good, mature, and easy to follow. But still, typos occur from time to time, showing a bit rush. For example, Section 5.1, “the x-axes denotes is the indices of layers” should remove “is”. Please make sure more proofreading will be done.


Over the passage of time Unmanned Autonomous Vehicles (UAVs), especially Autonomous flying drones grabbed a lot of attention in Artificial Intelligence. Since electronic technology is getting smaller, cheaper and more efficient, huge advancement in the study of UAVs has been observed recently. From monitoring floods, discerning the spread of algae in water bodies to detecting forest trail, their application is far and wide. Our work is mainly focused on autonomous flying drones where we establish a case study towards efficiency, robustness and accuracy of UAVs where we showed our results well supported through experiments. We provide details of the software and hardware architecture used in the study. We further discuss about our implementation algorithms and present experiments that provide a comparison between three different state-of-the-art algorithms namely TrailNet, InceptionResnet and MobileNet in terms of accuracy, robustness, power consumption and inference time. In our study, we have shown that MobileNet has produced better results with very less computational requirement and power consumption. We have also reported the challenges we have faced during our work as well as a brief discussion on our future work to improve safety features and performance.

The main context for this paper is two recent publications: Giusti et al.’s "A Machine Learning Approach to Visual Perception of Forest Trails for Mobile Robots” (2016) and Smolyanskiy et al.’s "Toward Low-Flying Autonomous MAV Trail Navigation using Deep Neural Networks for Environmental Awareness” (2017).
Giusti introduced a dataset of trail images (later called the “IDSIA dataset”) acquired by having a hiker wear three head-mounted cameras. The forward facing image is associated with a label “go straight”, whereas the two side images are associated with labels for “go left” and “go right”. Giusti then trained a convolutional neural network to predict these labels and used the network to guide a "quadrotor micro aerial vehicle”.
Smolyanskiy improves on Giusti’s work by (1) gathering additional trail image data using three cameras mounted to face forward but with lateral offsets and (2) using this additional data to train a 6 output neural network (“Trailnet”) which predicts both view orientation and lateral offset. In addition, they also combined predicted pose relative to the trail with predictions of localized objects and a depth map for potential obstacles. They compared several neural network architectures for predicting the view angle on the IDSIA data as well as the closed-loop performance of each network in avoiding collisions while operating within a UAV on a previously unseen trail. Though Trailnet did not achieve the highest accuracy (84% vs. the max 92% achieved by ResNet-18), it was the only network that achieved 100% collision avoidance on their UAV test course.
This paper, "A CASE STUDY ON OPTIMAL DEEP LEARNING MODEL FOR UAVS”, attempts to evaluate two potentially better convolutional neural networks for UAV trail guidance. They fine tune pertained Inception-Resnet and MobileNet models to predict the IDSIA dataset. These then both achieve better accuracy on the IDSIA test set and were analyzed for inference time and power consumption. These two models are then run through a single simulated path, where both seem to perform adequately across 2 turns in the path.
This paper has a variety of essential flaws.
1. A large portion of the text is devoted to their hardware and UAV control but they were not able to actually run models on a physical UAV "due to a hardware bug we were facing with the FCU”.
2. The paper claims to "introduce to the best of our knowledge, a very first comparative study of three algorithms in order to find a better motion control of a drone for detecting a trail”. This is a confusing claim since a comparison of neural network architectures is a central part of the evaluation in the Smolyanskiy paper.
3. The higher accuracy on view orientation does not seem relevant since it was also achieved by Smolyanskiy et al. with networks that they then showed performed worse when combined with object detection, obstacle depth inference and combined controller.
4. The sentence "An important goal of our method is to demonstrate the effectiveness of low cost systems for the complex task of flying an autonomous drone” appears to have been plagiarized from “Learning to Fly by Crashing” (2017) which contains "A important goal of our method is to demonstrate the effectiveness of low cost systems for the complex task of flying in an indoor environment”.


There has been a recent trend in training neural networks to replace data structures that have been crafted by hand, with an aim for faster execution, better accuracy, or greater compression. In this setting, a neural data structure is instantiated by training a network over many epochs of its inputs until convergence. In many applications this expensive initialization is not practical, for example streaming algorithms -where inputs are ephemeral and can only be inspected a small number of times. In this paper we explore the learning of approximate set membership over a stream of data in one-shot via meta-learning. We propose a novel memory architecture, the Neural Bloom Filter, which we show can be more compressive than Bloom Filters and several existing memory-augmented neural networks.

SUMMARY
The paper proposes a neural network based architecture to solve the approximate set membership problem, in the distributional setting where the in-set and out-of-set elements come from two unknown and possibly different distributions.
COMMENTARY
The topic of the paper is interesting, and falls into the popular trend of enhancing classical data structures with learning algorithms. For the approximate set membership problem, this approach was already suggested by (Kraska et al. 2018) and studied further in (Mitzenmacher 2018a,b). The difference in the current paper is that the proposed approach relies on "meta-learning", apparently to facilitate online training and/or learning across multiple sets arising from the same distribution; this is what I gather from the introduction, even though as I write below, I feel this point is not properly explained.
My main issue with the paper is that its conceptual contribution seems limited and unclear. It suggests a specific architecture whose details seem mostly arbitrary, or at least this is the impression the reader is left with, as the paper does rather little in terms of discussing and motivating them or putting them in context. Moreover, since the solution ultimately relies on a backup Bloom Filter as in (Kraska et al. 2018), it is hard to not view it as just an instantiation of the model in (Kraska et al. 2018, Mitzenmacher 2018a) with a different plugging of learning component. It would help to flesh out and highlight what the authors claim are the main insights of the paper.
Another issue I suggest revising pertains to the writing. The problem setting is only loosely sketched but not properly defined. How exactly do different subsets coming into play? Specifically, the term "meta-learning" appears in the title and throughout the paper, but is never defined or explained. The authors should write out what exactly they mean by this notion and what role it plays in the paper. This is important since to my understanding, this is the main point of departure from the aforementioned recent works on learning-enhanced Bloom Filters.
The experiments do not seem to make a strong case for the empirical advantage of the Neural Bloom Filter. They show little to no improvement on the MNIST tasks, and some improvement on a non-standard database related task. One interesting thing to look at would be the workload partition between the learning component and the backup filter, meaning what is the rate of false negatives emitted by the former and caught by the latter, and how the space usage breaks down between them (vis-a-vis the formula in Appendix B). For example, it seems plausible that on the class familiarity task, the learning component simply learns to be a binary classifier for the chosen two MNIST classes and mostly ignores the backup filter, whereas in the uniform distribution setting, the learning component only memorizes a small number of true and false positives and defers almost the entire task to the backup filter. I am not sure what to expect on the intermediate exponential distribution task.
Other comments/questions:
1. For the classical Bloom Filter, do the results reported in the experimental plots reflect the empirical false-positive rate measured in the experiment, or just the analytic bound?
2. On that note, it is worth noting that the false positive rate of the classical Bloom Filter is different than the one you report for the neural-net based architectures. The Bloom Filter FP probability is over its internal randomness (i.e. its hash functions) and is independent of the distribution of queries, which need not be randomized at all. For the neural-net based architectures, the measured FP rate is w.r.t. a specific distribution of queries. See the discussion in (Mitzenmacher 2018a), sections B-C.
3. The works (Mitzenmacher 2018a,b) should probably at least be referenced in the related work section.
CONCLUSION
While I like the overall topic of the paper, I currently find the conceptual contribution to be too thin, raising doubts on novelty and significance. In addition, the presentation is somewhat lacking in clarity, and the practical merit is not well established. Notwithstanding the public nature of ICLR submissions, I would suggest more work on the paper prior to publication.
REFERENCES
M. Mitzenmacher, A Model for Learned Bloom Filters and Related Structures, 2018, see https://arxiv.org/pdf/1802.00884.pdf.
M. Mitzenmacher, Optimizing Learned Bloom Filters by Sandwiching, 2018, see https://arxiv.org/pdf/1803.01474.pdf.
(Update: score revised, see below.)


This paper proposes a novel approach to train deep neural networks by unlocking the layer-wise dependency of backpropagation training. The approach employs additional modules called local critic networks besides the main network model to be trained, which are used to obtain error gradients without complete feedforward and backward propagation processes. We propose a cascaded learning strategy for these local networks. In addition, the approach is also useful from multi-model perspectives, including structural optimization of neural networks, computationally efficient progressive inference, and ensemble classification for performance improvement. Experimental results show the effectiveness of the proposed approach and suggest guidelines for determining appropriate algorithm parameters.

This paper describes a method of training neural networks without update locking. The idea is a small modification on top of Czarnecki et al. Critic training, where instead of using final loss as a critic target, one bootstraps from critics on other layers. In particular, if only one module is present, these two approaches are actually identical. To be more precise, the only difference between these two methods is that (7) in Critic training would change to l(L_i, L_N). As a consequence, method becomes forward unlocked too. It is worth noting, that in the appendix of Czarnecki et al. it is shown that this particular method (critic training) under simple conditions actually "degenerates" to deep supervision (which is forward unlocked too). Consequently unlocking property as such is not a big contribution of the proposed method. Rest of the paper includes following elements:
- empirical evaluation showing improvement over critic training by 0.4% in CIFAR10 and 0.9% in CIFAR100 when using 3 splits.
- expansion on using the model for progressive inference.
Given standard deviation of errors in Table 1. it is not clear how significant these improvements are. How many samples were used to estimate these quantities? It is worth noting, that Critic training was showed to be outperformed by Sobolev Training in the same paper authors cite, but its performance is not reported despite looking like a well defined baseline. In particular, can these two methods be combined?
I believe that this is an interesting research direction, however paper in its current form seems as a small incremental improvement over sota, and could be significantly improved by for example:
- providing more comprehensive evaluation (including estimating accuracy to lower std errors)
- adding other baseline solutions (such as Sobolev training, cDNI, or deep supervision)
- considering any form of convergence/dynamics analysis of the proposed approach


MULTI-AGENT DUAL LEARNING
Dual learning has attracted much attention in machine learning, computer vision and natural language processing communities. The core idea of dual learning is to leverage the duality between the primal task (mapping from domain X to domain Y) and dual task (mapping from domain Y to X ) to boost the performances of both tasks. Existing dual learning framework forms a system with two agents (one primal model and one dual model) to utilize such duality. In this paper, we extend this framework by introducing multiple primal and dual models, and propose the multi-agent dual learning framework. Experiments on neural machine translation and image translation tasks demonstrate the effectiveness of the new framework. In particular, we set a new record on IWSLT 2014 German-to-English translation with a 35.44 BLEU score, achieve a 31.03 BLEU score on WMT 2014 English-toGerman translation with over 2.6 BLEU improvement over the strong Transformer baseline, and set a new record of 49.61 BLEU score on the recent WMT 2018 English-to-German translation.

Summary
The paper proposes to modify the "Dual Learning" approach to supervised (and unsupervised) translation problems by making use of additional pretrained mappings for both directions (i.e. primal and dual). These pre-trained mappings ("agents") generate targets from the primal to the dual domain, which need to be mapped back to the original input. It is shown that having >=1 additional agents improves training of the BLEU score in standard MT and unsupervised MT tasks. The method is also applied to unsupervised image-to-image "translation" tasks.
Positives and Negatives
+1 Simple and straightforward method with pretty good results on language translation.
+2 Does not require additional computation during inference, unlike ensembling.
-1 The mathematics in section 3.1 is unclear and potentially flawed (more below).
-2 Diversity of additional "agents" not analyzed (more below).
-3 For image-to-image translation experiments, no quantitative analysis whatsoever is offered so the reader can't really conclude anything about the effect of the proposed method in this domain.
-4 Talking about "agents" and "Multi-Agent" is a somewhat confusing given the slightly different use of the same term in the reinforcement literature. Why not just "mapping" or "network"?
-1: Potential Issues with the Maths.
The maths is not clear, in particular the gradient derivation in equation (8). Let's just consider the distortion objective on x (of course it also applies to y without loss of generality). At the very least we need another "partial" sign in front of the "\delta" function in the numerator. But again, it's not super clear how the paper estimates this derivative.  Intuitively the objective wants f_0 to generate samples which, when mapped back to the X domain, have high log-probability under G, but its samples cannot be differentiated in the case of discrete data. So is the REINFORCE estimator used or something? Not that the importance sampling matter is orthogonal. In the case of continuous data x, is the reparameterization trick used? This should at the very least be explained more clearly.
Note that the importance sampling does not affect this issue.
-2: Diversity of Agents.
As with ensembles, clearly it only helps to have multiple agents (N>2) if the additional agents are distinct from f_1 (again without loss of generality this applies to g as well). The paper proposes to use different random seeds and iterate over the dataset in a different order for distinct pretrained f_i. The paper should quantify that this leads to diverse "agents". I suppose the proof is in the pudding; as we have argued, multiple agents can only improve performance if they are distinct, and Figure 1 shows some improvement as the number of agents are increase (no error bars though). The biggest jump seems to come from N=1 -> N=2 (although N=4 -> N=5 does see a jump as well). Presumably if you get a more diverse pool of agents, that should improve things. Have you considered training different agents on different subsets of the data, or trying different learning algorithms/architectures to learn them? More experiments on the diversity would help make the paper more convincing.


We propose a class of very simple modifications of gradient descent and stochastic gradient descent. We show that when applied to a large variety of machine learning problems, ranging from softmax regression to deep neural nets, the proposed surrogates can dramatically reduce the variance and improve the generalization accuracy. The methods only involve multiplying the usual (stochastic) gradient by the inverse of a positive definitive matrix coming from the discrete Laplacian or its high order generalizations. The theory of Hamilton-Jacobi partial differential equations demonstrates that the implicit version of new algorithm is almost the same as doing gradient descent on a new function which (i) has the same global minima as the original function and (ii) is "more convex". We show that optimization algorithms with these surrogates converge uniformly in the discrete Sobolev H p σ sense and reduce the optimality gap for convex optimization problems. We implement our algorithm into both PyTorch and Tensorflow platforms which only involves changing of a few lines of code. The code will be available on Github.

The paper considers SGD with a scaled norm; in the non-stochastic case (first equation in section 2.1), it is gradient descent in a fixed non-Euclidean norm, but it is the stochastic case that is most interesting. The paper connects this, somewhat, to a Hamilton-Jacobi equation, but then relaxes the implicit step to an explicit step.
There is solid theory (Prop 2, 3 and 4) for convergence, which makes sense since this is the same as usual SGD but in a different Hilbert space. Since the inner product is stationary, it's just a fixed Hilbert space, so any convergence proofs that work for arbitrary Hilbert space immediately give the result.
The computational experiments are impressive, and demonstrate a lot of competence with modern neural nets. Some results are hard to interpret (Figs 9, 10) though.
As for why use the Laplacian, Prop 8 (combined with Prop 6) gives some idea: that we lower the variance, without cheating (ie., we could trivially lower the variance by just multiplying by a small number, but because the operator preserves the sum of the components, it is not "cheating").  That is helpful, though it doesn't give a complete picture yet.  The explanation about the link to the "more convex" function I find completely inaccurate and misleading (see technical comments for why).
The writing is mainly fine, though some sentences are written poorly and would benefit from a revision, e.g., 2nd paragraph, "But none of them is suitable to train deep neural nets (DNNs)." is quite awkward [also, in this sentence, please explain *why* they are not suitable!]
The paper circumvents the page limit by using a smaller font (starting on page 3). This might seem like a minor issue, but it is violating the page limit, and not fair to other papers (unless I have misunderstood; the meta-reviewers can probably comment about this).  I do not think it would be unfair to reject the paper on these grounds. It leaves a bad taste in my mouth after reading the paper.
Technical comments:
- page 1, this is called a "tri-diagonal" linear system, but it is not, it is circulant due to the upper-right and lower-left entries (the authors are well-aware of this, but the reader maybe confused; especially since if it were tri-diagonal, it would be inverted via the Thomas algorithm not the FFT).
- Section 2: my first impression on reading this is that you've re-discovered the proximal point envelope and the Moreau envelope (and, looking at the proof of Prop. 1, the authors are aware of this connection).  In this context, it's not clear why A_sigma is helpful, as opposed to any positive definite matrix.
- The actual statement of Proposition 1 is unclear. What does "the ... update ... permits .." mean? i.e., "permits" is a weird, vague choice of words. What are you actually proving?
- Section 2.1 moves from the proximal point method (in a scaled norm) to the gradient descent method (in a scaled norm). Clearly, these two methods are different, and just as in ODE schemes, the implicit version is unconditionally stable while the explicit one isn't. So motivating your method by "smoothing" or "adding convexity" is really misleading. You could define equation (2) and the u(w,t) equation by replacing A_sigma with the identity, and as long as tau > 0, this also "convexifies", but then if you go from implicit to explicit, you get regular GD, so you haven't really done anything.  So, I do not buy this connection that your method "convexifies" the function.
- Using the FFT to invert seems slow (the theoretical flop-count is good, but it's still super-linear, and requires global data movement, so not good for a distributed implementation).  If you really did define A to be tri-diagonal, then you could invert naively in a linear time algorithm with local data movement. Why not use a tri-diagonal A? It might not satisfy prop 8 exactly, but it'd be close, and a lot faster in practice.  From a "finite-difference" point-of-view, I don't see an inherent argument about why you want circular boundary conditions.
- Remark 1 seems out-of-place. Why is that included?
- Section 3.2, and Fig. 5.  It's not clear that the improved generalization results are due to broader local minima, or if it's because the methods converged faster on the training data (since they were limited to 200 epochs). Showing the training error, as a function of epoch, would help clarify. Similar comment for other experiments too.
- Section 4 was impressive in the implementations. Nice work.
- The acknowledgments used the boiler-plate latex template text.
** summary **
Quality: Good
Clarity: OK
Originality: mixed
Significance: maybe high?


We propose a method which can visually explain the classification decision of deep neural networks (DNNs). There are many proposed methods in machine learning and computer vision seeking to clarify the decision of machine learning black boxes, specifically DNNs. All of these methods try to gain insight into why the network "chose class A" as an answer. Humans, when searching for explanations, ask two types of questions. The first question is, "Why did you choose this answer?" The second question asks, "Why did you not choose answer B over A?" The previously proposed methods are either not able to provide the latter directly or efficiently. We introduce a method capable of answering the second question both directly and efficiently. In this work, we limit the inputs to be images. In general, the proposed method generates explanations in the input space of any model capable of efficient evaluation and gradient evaluation. We provide results, showing the superiority of this approach for gaining insight into the inner representation of machine learning models.

The paper proposes an approach to provide contrastive visual explanations for deep neural networks -- why the network assigned more confidence to some class A as opposed to some other class B. As opposed to the applicability of previous approaches to this problem -- the approach is designed to directly answer the contrastive explanations question rather adapting other visual saliency techniques for the same. Overall, while I find the proposed approach simple -- the paper needs to address some issues regarding the claims made and should provide more quantitative experimental results justifying the same.
- Apart from some flaws in the claims made in the paper, the paper is easy to follow and understand.
- Assuming the availability of a latent model over the images of the input distribution, the proposed approach is directly applicable and faster.
- The authors clearly highlight the problems associated with existing explanation modalities and approaches; ranging from ones applicable to only specific deep architectures to ones using backpropagation based heuristics.
- The proposed approach to generate contrastive explanations is simple and is structured along the lines of methods utilizing probe images to explain decisions -- except for the added advantage that the provided explanations are instance-agnostic due to the assumption of a latent model over the input distribution.
Comments:
- One of the problems highlighted in the paper regarding existing explanation modalities is the use of another black-box to explain the decisions of an existing deep network (also somewhat of a black-box) which the authors claim their model does not suffer from. The proposed approach provides explanations by operating in the latent space of a learned generative model of the input distribution. The learned generator in itself is somewhat of a black-box itself -- there has been prior work indicating how much of the input distribution are GANs able to capture. As such, conditioning on a generative model to propose such contrastive explanations is to some extent using another black-box (generator) to explain the decisions of an existing one. Thus, the above claim made in the paper does not seem well-founded. Furthermore, in experiments, the paper does not provide any quantitatively convincing results to suggest the generator in use is a good one.
- While the authors suggest that a latent model over the input distribution needs to be trained only once and is applicable off-the-shelf for any further contrastive explanations regarding any network operating on the same dataset -- learning such a model of the input space is an overhead in itself. In this light, experiments demonstrating comparisons between GANs and VAEs as the reference generative model for explanations would have made the paper stronger (as the proposed approach relies explicitly on how good the generative model is).
- The paper proposes an interesting experiment to show that the proposed approach is somewhat capable of capturing slightly adversarial biases in the input domain (adding square to the top-left of images of class ‘8’). While I like this experiment, I feel this has not been explored to completion in the sense of experimenting with robustness with respect to structured as well as unstructured perturbations.
- Typographical Errors: Section 3.1 repeats the use of D for a discriminator as well as the input distribution. Procedure 1 and Procedure 2 share the same titles -- which is slightly misleading. In addition, Procedure 1 is not referenced in the text which makes is hard to understand the utility of the same. In Section 4.1, the use of Gradcam and Lime to generate counterfactual explanations is not very clear and makes it slightly hard to follow. Citations used for Gradcam are wrong -- Sundarajan et al., 2016 should be changed to Selvaraju et al., 2017.
Experimental Issues:
- Experimental results are provided only on MNIST and Fashion-MNIST. Since the paper focuses explicitly on providing contrastive explanations for choosing a class A over another class B -- experiments on datasets which do not have real-images seem insufficient. Additional experiments on at least ImageNet would have made the paper stronger.
Regarding contrastive explanations, experiments on datasets where distractor classes (y_probe) are present in addition to the class interest (y_true) seem important -- PASCAL VOC, COCO, etc. Specifically, since the explanations provided are visual saliency maps the paper would have been stronger if there were experiments suggesting -- what needs to change in a region of an image classified as a ‘cat’ to be classified as a ‘dog’ while there is an instance of the class - ‘dog’ present in the image itself. Also, section 7 in Gradcam (https://arxiv.org/pdf/1610.02391.pdf) provides a procedure to generate counter-factual explanations using Gradcam. Is there a particular reason the authors did not choose to adopt the above technique as a baseline?
- Experimental results provided in the paper are only qualitative -- as such, I do not find the comparisons (and improvements) over the existing approaches convincing enough. Since, there is no clear metric to evaluate contrastive explanations -- human studies to judge the class-discriminativeness (or trust) of the proposed approach would have made the paper stronger.
The authors adressed the issues raised/comments made in the review. In light of my comments below to the author responses -- I am not inclined towards increasing my rating and will stick to my original rating for the paper.


Many of the zero-shot learning methods have realized predicting labels of unseen images by learning the relations between images and pre-defined class-attributes. However, recent studies show that, under the more realistic generalized zero-shot learning (GZSL) scenarios, these approaches severely suffer from the issue of biased prediction, i.e., their classifier tends to predict all the examples from both seen and unseen classes as one of the seen classes. The cause of this problem is that they cannot properly learn a mapping to the representation space generalized to the unseen classes since the training set does not include any unseen class information. To solve this, we propose a concept to learn a mapping that embeds both images and attributes to the shared representation space that can be generalized even for unseen classes by interpolating from the information of seen classes, which we refer to shared manifold learning. Furthermore, we propose modality invariant variational autoencoders, which can perform shared manifold learning by training variational autoencoders with both images and attributes as inputs. The empirical validation of well-known datasets in GZSL shows that our method achieves the significantly superior performances to the existing relationbased studies.

The paper proposes an approach to generalized zero-shot learning by learning a shared latent space between the images and associated class-level attributes. To learn such a shared latent space and mapping for the same which is generalized and robust -- the authors propose ‘modality invariant variational autoencoders’ -- which allows one to perform shared manifold learning by training VAEs with both images and attributes as inputs. Empirical results demonstrate improvements over existing approaches on the harmonic mean metric present in the generalized zero-shot learning benchmark. Other than the concerns mentioned below, I like the basic idea adopted in the paper to extend Vedantam et. al. (2018)’s joint-VAEs (supporting unimodal inference) to the framework of generalized zero-shot learning. The proposed approach clearly results in improvements over baselines and existing approaches.
Comments:
- A minor correction. The paper claims the bias towards seen classes at inference for the existing GZSL approaches is due to the inability of obtaining training data for the unseen classes. In my opinion, this should be rephrased as the inability to learn a generalized enough representation (joint or otherwise) that is aware of the shift in distribution from seen to unseen classes (images or attributes) as this information is not available apriori.
- Writing Clarity Issues. In general, there is significant repetitions along certain lines throughout the introduction and approach. While the paper overall does a good job of explaining the motivation as well as the approach, some of the sections (and sentences within) could be written better to express the point being made. Specifically, the first paragraph in the introduction seems to be structured more from a few-shot setting. The paper would benefit from talking about few-shot learning first and then extending to the extreme setting of zero-shot learning. Similarly, the second paragraph in the introduction could be written more succinctly to express the point being made. The sentences -- “Moreover, it is difficult….widely available” -- are difficult to understand. Tables 4 and 5 should be positioned after the references section.
- A point repeatedly made in the paper suggests that learning unidirectional mappings from images to attributes (or otherwise) suffers from generalization to unseen classes. While I agree with this statement, most methods in GZSL hold out a subset of seen classes as validation (unseen) classes while learning such a mapping -- which I believe was also being done while learning the joint model in MIVAE (Can the authors confirm this? Is yes, how were these classes chosen?). As such, the authors should stress on the advantages learning a joint latent model over both modalities offers as opposed to unidirectional mappings while mentioning the above points.
- Learning the joint latent space for images and attributes has been referred to as learning a shared manifold in the paper -- with associated terms such as manifold representation being used as well. Sharing a latent space need not imply learning an entire manifold as the subspace captured by the latent space might as well be localized in the manifold in which it exists. Can the authors comment more on this connection with respect to the points around “shared manifold learning”?
- During inference, the authors operate in the latent space to find the most-relevant class by enumerating over all classes the KL-divergence between the unimodal encoder embeddings. Is there a particular reason the authors chose to operate in the latent space as opposed to operating in a modality space? Specifically, given an image the authors could have used the p(a|z) decoder to infer the attribute given the encoded z -- and subsequently finding the 1-nearest neighbor in that space. Any reason why this approach was not adopted?
- On page 4, regarding the term L_dist in the objective for MIVAE, the authors draw the connections made in the appendix of Vedantam et. al. (2018) regarding the minimization of KL-divergence between the bimodal and a unimodal variational posterior(s). While the connection being made is accurate, the subsequent solution modes identified in the following paragraph -- “When equation 2 becomes minimum…” -- do not seem accurate. At minimality, unimodal encoders should be equivalent to the bimodal encoder marginalized over the absent rv under the conditional distribution of the data. Could the authors comment on whether the version presented in the paper is intended or is merely a typographical mistake?
Section 5 experiments suggest the learning rate used in practice was 10^3. Assuming a typo, this should be presumably 10^-3.
Experimental Issues.
- The authors should explicitly mention if they are using the proposed split throughout all baselines and approaches for GZSL evaluations. It’s not explicitly mentioned in the text and is an important detail that should not be left out. Only the appendix mentions the number of seen/unseen classes.
- How did the authors select a validation split (held out seen classes) to train MIVAE? Did they directly borrow the training and validation splits present in the proposed split? Or did they create a split of their own? If latter, how was the split created? In general, I am curious about how the MIVAE checkpoint for inference was chosen.
- In section 5.2, the reasons in the 3rd paragraph elaborating \lambda_map=1 vs 0 not being too different for AWA and aPY are not clear. Could the authors comment a bit more on them?
The authors adressed the issues raised/comments made in the review. In light of my comments below to the author responses -- I am not inclined towards increasing my rating and will stick to my original rating for the paper.


We propose a new class of probabilistic neural-symbolic models for visual question answering (VQA) that provide interpretable explanations of their decision making in the form of programs, given a small annotated set of human programs. The key idea of our approach is to learn a rich latent space which effectively propagates program annotations from known questions to novel questions. We do this by formalizing prior work on VQA, called module networks (Andreas, 2016) as discrete, structured, latent variable models on the joint distribution over questions and answers given images, and devise a procedure to train the model effectively. Our results on a dataset of compositional questions about SHAPES (Andreas,  2016)  show that our model generates more interpretable programs and obtains better accuracy on VQA in the low-data regime than prior work.

The paper presents a new approach for performing visual query answering. System responses are programs that can explain the truth value of the answer.
In the paper, both the problems of learning and inference are taken into account.
To answer queries, this system takes as input an image and a question, which is a set of word from a given vocabulary. Then the question is modeled by a plan (a series of operation that must be performed to answer the query).  Finally, the found answer with the plan are returned. To learn the parameters of the model, the examples are tuples composed by an image, a question, the answer, and the program.
Experiments performed on the SHAPES dataset show good performance compared to neural model networks by Johnson et al.
The paper is well written and clear. I have not found any specific problems in the paper, the quality is high and the approach seems to me to be new and worth studying.
The discussion on related work seems to be good, as well as the discussion on the results of the tests conducted.
On page 5, in equation (3) it seems to me that something is missing in J. Moreover, In Algorithm 1, in lines 4 and 9, the B after the arrow should be written in italic.
Overall, there are several typos that must be corrected. I suggest a double check of the English. For example:
- page 3, "as modeling *uncertaintly* should..."
- page 6, "Given this goal, we *consrtuct* a latent *varible* ..."
- page 8, in paragraph "Effect of optimizing the true ELBO", the word "that" is repeated twice in the 3rd row
- page 13, "for the" repeated twice in "Moving average baseline" paragraph. Also, in the last line of this paragraph, the sentence seems incomplete.
Pros
- The results are convincing
- The approach is clearly explained
Cons
- English must be checked


Infinite length of trajectories is an almost universal assumption in the theoretical foundations of reinforcement learning. In practice learning occurs on finite trajectories. In this paper we examine a specific result of this disparity, namely a strong bias of the time-bounded Every-visit Monte Carlo value estimator. This manifests as a vastly different learning dynamic for algorithms that use value predictors, including encouraging or discouraging exploration. We investigate these claims theoretically for a one dimensional random walk, and empirically on a number of simple environments. We use GAE as an algorithm involving a value predictor and evolution strategies as a reference point.

The paper addresses the problem of truncating trajectories in Reinforcement Learning. The scope is right for ICLR.
The presentation is pretty good as far as the English goes but suffers from serious problems on the level of formulating concepts. Overall, I believe that the issue addressed by the paper is important, but I am not sure whether the approach taken by the authors addresses it.
I have the following complaints.
1. The paper suffers from a fundamental confusion about what problem it is trying to address. There are two straightforward ways to formulate the problem. First, we can formulate the task as solving an MDP with infinite trajectories and ask the question of what we can learn training from finite ones. The learned policies would then be evaluated on the original MDP (in practice using trajectories that are, say, an order of magnitude longer or, for a simple MDP, analytically). Second, we can consider the family of episodic MDPs parametrised by trajectory length T and ask the question of what we can learn by training on some values of T and evaluating on others. These problems are similar, but not the same and should be carefully distinguished. Right now, the introduction reads like the authors were trying to use the first approach but Section 4 reads like they are doing the second: "If during training we just used times up to T, but deployed the agent in an environment with times greater than T". Either way, the concept of bias, which appears throughout the paper, isn't formally defined anywhere. I believe that a paper that claims to address bias should have an equation that defines it as a difference between two clearly defined quantities. It should also be clearly and formally distinguished which quantities are deterministic and which are random variables. The introduction seems to (implicitly?) define bias as a random variable, section 3.1 seems to talk about "bias introduced by initializing the network". As the paper stands now, the working definition of bias used by the authors seems to be that some quantity is vaguely wrong. I do expect a higher standard of clarity in a scientific paper,
2. The paper confounds the problem of learning the value function, specifying the initial estimates of the value function and exploration. The analysis of exploration is entirely informal and suffers from the lack of clear problem formulation as per (1). Of course, one can influence exploration by initialising the value function in various ways, and this may respond differently to different truncations (different values of T), but I don't see how it is related to the "bias" problem that the paper is trying to address. In any case, I wish the authors either provided a formal handle on exploration or shift the focus of the paper and remove it,
3. I don't see what hypothesis the experiments are trying to test. Clearly, if I train my agent on a different MDP and test it on a different one, I get a mismatch. The lack of clear definitions as per (1) comes back with a vengeance.
4. Section 1.1 seems to exist purely to create a spurious impression of formality, which bears little relevance for what the paper is actually about. RL, as traditionally formulated, uses discrete time-steps so the Brownian motion model developed in this section doesn't seem very applicable - it is true that it is a limiting case of a family of discrete chains, but I don't see how this produces any insights about RL - chains are easy to simulate, so why not test on a chain directly? In any case, the result shown in Section 1.1 is entirely standard, can be found in any textbook on stochastic processes and was likely introduced purely to cover for the lack on any substantial theory that comes form the authors.
To summarize: if the authors address these points, there is a possibility that the ideas presented in this draft may somehow lead to a paper at some later point. However, I feel that the required changes would be pretty massive and don't see how the authors could make it during the ICLR revision phase - the problems aren't details or technicalities but touch the very substance of what the paper is trying to do. Basically, the whole paper would need to be re-written.
another minor point:  sloppy capitalization in section 2.1
===================
For reasons outlined in my comment below, I updated the score to 5 (for the new heavily updated version).


Deep neural networks with discrete latent variables offer the promise of better symbolic reasoning, and learning abstractions that are more useful to new tasks. There has been a surge in interest in discrete latent variable models, however, despite several recent improvements, the training of discrete latent variable models has remained challenging and their performance has mostly failed to match their continuous counterparts. Recent work on vector quantized autoencoders (VQ-VAE) has made substantial progress in this direction, with its perplexity almost matching that of a VAE on datasets such as CIFAR-10. In this work, we investigate an alternate training technique for VQ-VAE, inspired by its connection to the Expectation Maximization (EM) algorithm. Training the discrete autoencoder with EM and combining it with sequence level knowledge distillation alows us to develop a nonautoregressive machine translation model whose accuracy almost matches a strong greedy autoregressive baseline Transformer, while being 3.3 times faster at inference.

Summary:
This paper presents a new training algorithm for vector-quantized autoencoders (VQVAE), a discrete latent variable model akin to continuous variational autoencoders.
The authors propose a soft-EM training algorithm for this model, that replaces hard assignment of latent codes to datapoints with a weighted soft-assignment.
Overall the technical writing in the paper is sloppy, and the presentation of the generative model takes the form of an algorithmic description of the training algorithm, rather than being a clear definition of the generative model itself.
The technical presentation of the work by the authors starts only at page 5 (taking less than a full page), after several pages of imprecise presentation of previous and related work. The paper could be significantly improved by making this preceding material more concise and rigorous.
Quantitative experimental evaluation is limited to a machine translation task, which is rather uncommon in the literature on generative latent variable models. I would expect evaluation in terms of held-out data log-likelihood (ie bits-per-dimension) used in probabilistic generative models, and possibly also using measures from the GAN literature such as inception scores. Datasets that are common include CIFAR-10 and resized variants of the imagenet dataset.
Specific comments:
- Please adhere to the ICLR template bibliography style, which is far more readable than the style that you used.
- Figure 1 does not seem to be referenced in the text.
- The last paragraph of section 2.1 is unclear. It mentions a sampling a sequence of latent codes. The notion of sequentiality has not been mentioned before, and it is not clear what it refers to in the context of the model defined so far up to that point.
- The technical notation is very sloppy.
* In numerous places the paper refers to the joint distribution P(x1,…,x_n, z1, …, zn) without defining that the distribution factorizes across the samples (xi,zi), and without specifying the forms of p(zi) and p(xi|zi).
* This makes that claims such as “computing the expectation in the M step (Equation 11) is computationally infeasible” are not verifiable.
- Please be clear about how much is gained by replacing the exact M-step with a the one based on the samples from the posterior computed in the E-step.
- What is the reason to decode the weighted average of the embedding vectors, rather than decoding all of them, and updating the decoder in a weighted manner?
- reference 14 for Variational autoencoders is incorrect, please use the following citation instead:
@InProceedings{kingma14iclr,
  Title                    = {Auto-Encoding Variational {B}ayes},
  Author                   = {D. Kingma and M. Welling},
  Booktitle                = {{ICLR}},
  Year                     = {2014}
}
- The related work section (4) provides a rather limited overview of relevant related work.
Half of it is dedicated to recent advances in machine translation, which does not bear a direct connection to the technical material presented in section 3.
- There is no justification of using *causal* self-attention on the source embedding, is this a typo?
- As for the experimental evaluation results: it seems that distillation is a much more critical factor to achieve good performance than the proposed EM training of the VQ-VAE model. Unfortunately, this fact goes unmentioned when discussing the experimental results.
- What is the significance of the observed differences in BLEU scores? Please report average performance and standard deviations over several runs with randomized parameter initialization and batch scheduling.
- It seems that the tuning of the number of discrete latent codes (table 2 in appendix) and other hyper-parameters (table 3 in appendix) was done on the test set, which is also used to compare to related work. A separate validation set should be used for hyper parameter tuning in machine learning experiments.
- It seems that all curves in figure 3 collapse from about 45 BLEU to values around 17 BLEU, why is this? The figure is hard to read since poor quality, and curves that are superposed.


We propose Adaptive Sample-space & Adaptive Probability (ASAP) coding, an efficient neural-network based method for lossy data compression. Our ASAP coding distinguishes itself from the conventional methods based on adaptive arithmetic coding in that it models the probability distribution for the quantization process in such a way that one can conduct back-propagation for the quantization width that determines the support of the distribution. Our ASAP also trains the model with a novel, hyper-parameter free multiplicative loss for the rate-distortion tradeoff. With our ASAP encoder, we are able to compress the image files in the Kodak dataset to as low as one fifth the size of the JPEG-compressed image without compromising their visual quality, and achieved the state-of-the-art result in terms of MS-SSIM based rate-distortion tradeoff.

The paper proposes Adaptive Sample-space & Adaptive Probability (ASAP) coding for image compression based on neural networks. In contrast to most prior methods, which adhere to a fixed quantization scheme (i.e. with fixed number of quantization levels, and fixing the level themselves), the proposed method jointly learns a probability model of the quantized representation (the bottleneck of an autoencoder model) for coding and a corresponding adaptive quantization scheme. The distribution of each entry in the bottleneck before quantization is modeled as a Gaussian, whose mean and variance are predicted by a neural network conditionally on bottleneck entries on a grid at different scales (similar as in Nakanishi et al. 2018). The same network also predicts quantization intervals to adaptively quantize the respective entry of the bottleneck. Together, the predicted means, variances, and quantization intervals are used to obtain an estimate of the code length. The proposed compression networks are trained with a novel multiplicative loss, showing clear improvements over prior methods Rippel & Bourdev 2017, Nakanishi et al. 2018 on the Kodak and Raise1k data sets in terms of MS-SSIM.
Pros:
The results presented in this paper seem to be state-of-the-art, and innovation on quantization, which has not attracted a lot of attention in the context of neural network-based image compression is a welcome contribution. The method also seems to outperform the recent method [1], which should be included for comparison.
Questions:
A major issue is, however, that it is unclear from the results whether the gains are due to the novel quantization system, or due to the novel loss. From Fig. 7 it looks like the loss BPP + \lambda (1-MS-SSIM) (assuming the formula in (6)) is correct, and the legend in Fig. 7 incorrect) that is used in most other works performs essentially on par with  Rippel & Bourdev 2017, Nakanishi et al. 2018. For example at 1 bpp, this loss yields an MS-SSIM of 0.992 which is essentially the same as Rippel & Bourdev 2017, Nakanishi et al. 2018 obtain, cf. Fig. 2. To show that the improvement is due to the learned quantization and not just because of the loss (8) an ablation experiment should be done. One could e.g. train the proposed method with the same predictor, but without the employing the learned quantization scheme, and compare to the results obtained for the proposed method.
Furthermore, a better understanding of the loss (8) would be desirable. How is the MSE factor justified?
Also, it would be good to present visual examples at rates 0.1-1.0 bpp. All visual examples are at rates below 0.08 bpp, and the proposed method is shown to also outperform other methods at much higher rates.
[1] Ballé, J., Minnen, D., Singh, S., Hwang, S.J. and Johnston, N. Variational image compression with a scale hyperprior. ICLR 2018.


Recent advancements on Generative Adversarial Network (GAN) have inspired a wide range of works that generate synthetic images. However, current processes have to generate an entire image at once, and therefore resolutions are limited by memory or computational constraints. In this work, we propose COnditional COordinate GAN (COCO-GAN), which generates a specific patch of an image conditioned on a spatial position rather than the entire image at a time. The generated patches are later combined together to form a globally coherent full-image. With this process, we show that the generated image can achieve competitive quality to state-of-the-arts and the generated patches are locally smooth between consecutive neighbors. One direct implication of the COCO-GAN is that it can be applied onto any coordinate systems including the cylindrical systems which makes it feasible for generating panorama images. The fact that the patch generation process is independent to each other inspires a wide range of new applications: firstly, "Patch-Inspired Image Generation" enables us to generate the entire image based on a single patch. Secondly, "Partial-Scene Generation" allows us to generate images within a customized target region. Finally, thanks to COCO-GAN's patch generation and massive parallelism, which enables combining patches for generating a full-image with higher resolution than state-of-the-arts.

The paper describes a GAN architecture and training methodology where a generator is trained to generate "micro-" patches, being passed as input a latent vector and patch co-ordinates. Micro-patches generated for different adjacent locations with the same latent vector are combined to generate a "macro" patch. This "macro" output is trained against a discriminator that tries to label this output as real and fake, as well as predict the location of the macro patch and the value of the latent vector. The generator is trained to fool the discriminator's label, and minimize the error in the prediction of location and latent vector information.
- The paper proposes a combination of different interesting strategies. However, a major drawback of the method is that it's not clear which of these are critical to the quality of the generated output.
- Firstly, it isn't clear to me why the further breakdown of the macro patch into micro patches is useful. There appears to be no separate loss on these intermediate outputs. Surely, a DC-GAN like architecture with sufficient capacity would be as well able to generate "macro" patches. The paper needs to justify this split into micro patches with a comparison to a direct architecture that generates the macro patches (everything else being the same). Note that applications like "interpolation" of micro patches could be achieved simply by interpolating crops of the macro patch.
- As a means of simply producing high-resolution images, it appears that "PGGAN" performs better than the proposed method. Therefore, the paper doesn't clearly explain the setting when the division into patches produces a better result. It is worth noting that the idea of applying "local" critics (i.e., discriminators acting on sub-regions) isn't new (e.g., Generative Image Inpainting with Contextual Attention in CVPR 2018). What's new is the proposed method's way of achieving consistency between different regions by providing the 'co-ordinate' of the patch as input (and seeking consistency in the latent vector through a loss)---rather than applying a discriminator at a coarser level on the downsampled image. But given the poorer performance compared to PGGAN, it isn't clear that there is a an advantage to this approach.
Overall, the paper brings up some interesting ideas, but it doesn't motivate all its design choices, and doesn't make a clear argument about the settings in which the proposed method would provide an actual advantage.
===Post-rebuttal
I'm upgrading my score from 5 to 6, because some of the ablation experiments do make the paper stronger. Having said that, I still think this is a borderline paper. "Co-ordinate conditioning" is an interesting approach, but I think the paper still lacks convincing experiments for its main motivating use case: generating outputs at a resolution that won't fit in memory within a single forward pass. (This motivation wasn't clear in the initial version, but is clearer now).
The authors' displayed some high-resolution results during the rebuttal phase, but note that they haven't tuned the hyper-parameter for these (and so the results might not be the best they can be). Moreover, they scale up the sizes of their micro and macro patches so that they're still the same factor below the full image. I think a version of this paper whose main experimental focus is on high-resolution data generation, and especially, from much smaller micro-macro patches, would make a more convincing case.
So while the paper is about at the borderline for acceptance, I do think it could be much stronger with a focus on high-resolution image experiments (which is after all, forms its motivation).


We study a simple generic framework to address the issue of bad training data; both bad labels in supervised problems, and bad samples in unsupervised ones. Our approach starts by fitting a model to the whole training dataset, but then iteratively improves it by alternating between (a) revisiting the training data to select samples with lowest current loss, and (b) re-training the model on only these selected samples. It can be applied to any existing model training setting which provides a loss measure for samples, and a way to refit on new ones. We show the merit of this approach in both theory and practice. We first prove statistical consistency, and linear convergence to the ground truth and global optimum, for two simpler model settings: mixed linear regression, and gaussian mixture models. We then demonstrate its success empirically in (a) saving the accuracy of existing deep image classifiers when there are errors in the labels of training images, and (b) improving the quality of samples generated by existing DC-GAN models, when it is given training data that contains a fraction of the images from a different and unintended dataset. The experimental results show significant improvement over the baseline methods that ignore the existence of bad labels/samples.

The authors propose an iterative method for discarding outlying training data: first, learn a model on the entire training dataset; second, identify the training examples that have high loss under the learned model; and then alternate between re-learning the model on the training examples that do not have high loss, and re-identifying the training examples with high loss under the new model. This method works for both supervised and unsupervised learning, and the authors show that in theory, their method has some convergence properties in the mixed linear regression and Gaussian mixture model settings. The authors also run some experiments on neural networks and datasets with synthetic noise to show the benefits of their proposed method.
The problem of noisy datasets is relevant to almost all machine learning problems in the real world, and the authors' method shows promise as a straightforward way to increase performance on such noisy data. However, my opinion is that the authors need to make a stronger case, theoretically and/or experimentally, for why their method should be preferred to other methods. Detailed comments follow.
== Experiments ==
1) No comparisons are provided to other outlier detector methods (e.g., based on nearest neighbors, distance to centroid, influence functions, etc.) or techniques that also purport to deal with noisy labels (e.g., by modifying the learning algorithm or loss function). While there are too many existing methods to expect the authors to benchmark against all of them, it's important to at least have a couple of representative comparisons.
2) It'd be nice to have an ablative analysis to tease out the factors behind the gain in accuracy. For example, is the iteration important, or would a single pass suffice? How robust is the algorithm to tau, the fraction of data to discard? (The authors do test initializing randomly vs. initializing on the full dataset.)
3) The systematic label noise scenario seems to favor the authors' method (though the authors claim that it is a harder scenario than random label noise). It'd be helpful to see if the method works against random noise.
== Theory ==
4) The assumptions seem very restrictive. For example, for mixed linear regression (section 4), the features of all examples are assumed to be drawn i.i.d. from an isotropic Gaussian (so even the bad samples are drawn from the same distribution as the good samples; and all features are independent). To my knowledge, this is not a "standard and widely studied" assumption. For the Gaussian mixture model (section 5), a similar isotropic Gaussian assumption is made for each mixture.
5) Beyond the independence assumptions mentioned above, the initialization results make additional assumptions on the "bad" data (e.g., average distance of the good vs. bad parameters) that I found hard to parse. How strong are these assumptions? Do they hold on real datasets?
6) The convergence results (Theorems 1 and 3) have a constant term sigma in them. This is surprising and seems to me to considerably weaken the result -- one would expect that the dependence on sigma will decrease with n.
I think either a strong experimental or strong theoretical section would be sufficient for me to recommend acceptance. However, the paper currently shows potentially interesting experimental/theoretical results but does not do a comprehensive job of either side.


Context Aware Advertisement (CAA) is a type of advertisement appearing on websites or mobile apps. The advertisement is targeted on specific group of users and/or the content displayed on the websites or apps. This paper focuses on classifying images displayed on the websites by incremental learning classifier with Deep Convolutional Neural Network (DCNN) especially for Context Aware Advertisement (CAA) framework. Incrementally learning new knowledge with DCNN leads to catastrophic forgetting as previously stored information is replaced with new information. To prevent catastrophic forgetting, part of previously learned knowledge should be stored for the life time of incremental classifier. Storing information for life time involves privacy and legal concerns especially in context aware advertising framework. Here, we propose an incremental classifier learning method which addresses privacy and legal concerns while taking care of catastrophic forgetting problem. We conduct experiments on different datasets including CIFAR-100. Experimental results show that proposed system achieves relatively high performance compared to the state-of-the-art incremental learning methods.

The paper adresses the problem of incremental learning when data from new classes are available as a stream and one wants to be able to update to learn new observed classes without forgetting the older ones. There is a budget issue here and one does not want to just keep the whole training set of all known previously observed classes but rather one wants to consider a maximum memory budget allowed to store what is necessary for an optimal incremental learning (typical examples, statistics etc). There is also a privacy issue preventing from storing original training samples.
This is a relevant problem that is has gain interest in the last few years. It is related to topics such as few shot learning and meta few shot learning (with respect ti the number of examples per class that are kept, which is limited) and somehow to budget learning . Yet these topics and associated references are surprisingly not evoked in the text.
The paper is rather well written but it strongly lacks precision about the proposed method. A description of the ICARL state of the art method is missing and would have been mandatory since the proposed work appears to build on iCARL method. Actually the description of the method is very short since the dedicated section (§4) is mainly used to describe a rather standard convolutional auto encoder architecture. At the end one tries to guess what the proposed method consists in. As far as i understand it is based on iCARL method where selected examples of past observed classes are not stored as is but in their encoded form (by the convolutional autoencoder). At the end my understanding of the proposed approach is that it consists in an incremental progress of a state of the art method, then an incremental work with limited innovation.
By the way i am not sure of the meaning of pseudo exemplar as used in the proposed method. Are these drawn following a distribution computed on training samples ? Or are these pseudo exemplar because you use reconstructed samples from encodings (by the CAE).
When looking at experimental results the proposed method seem to bring some benefit but it does not look fully convincing. As written in the paper the proposed system outperforms iCARL in case the examples are encoded in the same dimension as original examples (hence no benefit on the storage side) but reaches similar performance when using less storage capacity.


In this paper, we focus on a new Named Entity Recognition (NER) task, i.e., the Multi-grained NER task. This task aims to simultaneously detect both finegrained and coarse-grained entities in sentences. Correspondingly, we develop a novel Multi-grained Entity Proposal Network (MGEPN). Different from traditional NER models which regard NER as a sequential labeling task, MGEPN provides a new method that proposes entity candidates in the Proposal Network and classifies entities into different categories in the Classification Network. All possible entity candidates including fine-grained ones and coarse-grained ones are proposed in the Proposal Network, which enables the MGEPN model to identify multi-grained entities. In order to better identify named entities and determine their categories, context information is utilized and transferred from the Proposal Network to the Classification Network during the learning process. A novel Entity-Context attention mechanism is also introduced to help the model focus on entity-related context information. Experiments show that our model can obtain state-of-the-art performance on two real-world datasets for both the Multi-grained NER task and the traditional NER task.

This paper proposed a entity proposal network for named entity recognition which can be effectively detect overlapped spans. The model obtain good performance on both Multi-grained NER task and traditional NER task. The paper is in general well written, the idea of proposal network break the traditional framework of sequence tagging formulation in NER task and thus can be effectively applied to detect overlapped named entities.
However, I still have many concerns regarding the notation, the novelty of the paper, and the comparison with related literature, especially on previous overlapped span detection NER papers. The detailed concerns and questions are as follows:
The notations are very confusing. Many of the notations are not defined. For example, what does $T$ in $2D_sl*2T$ below Eq. 4 indicates?  What does $R$ scores means? I guess $R$ does not equal to number of entity types, but I’m not sure what $R$ exactly indicates. If $R$ is not number of entity types, why do you need R scores for being an entity and R scores for not being an entity? And what is $t$ in Eq 5? Is that entity type id or something else?
I’m still confused how you select the entity spans from a large number of entity candidates. In Figure 5, if the max window length is 5, there may be more span candidates than the listed 5 examples, such as t_3 t_4 t_5. How do you prune it out?
Table 5 is weird. There is not comparison with any baselines but just a report of the performance with this system. I don’t know what point this table is showing.
This is not the first paper that enumerates all possible spans for NER task.The idea of enumerating possible spans for NER task has appeared in [1] and can also effectively detect overlapped span. I would like to see the performance comparison between the two systems. The enumerating span ideas has been applied in many other tasks as well such as coreference resolution [2]and SRL[3], none of which is mentioned in related work.
I feel that most of the gain is from ELMo but not the model architecture itself, since in Table 4, the improvement from the ELMo is only 0.06. The LSTM-LSTM-CRF is without adding ELMo, which is not a fair comparison.
The comparison of baselines is not adequate and is far from enough. The paper only compares with LSTM+CRF frameworks, which are not designed for detecting overlapped spans. There are many papers on detecting overlapping spans, such as [4], [5] and [6]. It’s important to compare with those paper since those methods are especially designed for overlapped span NER tasks.
[1] Multi-Task Identification of Entities, Relations, and Coreferencefor Scientific Knowledge Graph Construction, EMNLP 2018
[2] End-to-end neural coreference resolution, EMNLP 2017
[3] Jointly predicting predicates and arguments in neural semantic role labeling, ACL 2018
[4] Nested Named Entity Recognition Revisited, NAACL 2018
[5] A Neural Layered Model for Nested Named Entity Recognition, NAACL 2018
[6] Neural Segmental Hypergraphs for Overlapping Mention Recognition, EMNLP 2018


META-LEARNING FOR STOCHASTIC GRADIENT MCMC
Stochastic gradient Markov chain Monte Carlo (SG-MCMC) has become increasingly popular for simulating posterior samples in large-scale Bayesian modeling. However, existing SG-MCMC schemes are not tailored to any specific probabilistic model, even a simple modification of the underlying dynamical system requires significant physical intuition. This paper presents the first meta-learning algorithm that allows automated design for the underlying continuous dynamics of an SG-MCMC sampler. The learned sampler generalizes Hamiltonian dynamics with state-dependent drift and diffusion, enabling fast traversal and efficient exploration of energy landscapes. Experiments validate the proposed approach on learning tasks with Bayesian fully connected neural networks, Bayesian convolutional neural networks and Bayesian recurrent neural networks, showing that the learned sampler out-performs generic, hand-designed SG-MCMC algorithms, and generalizes to different datasets and larger architectures.

In the paper "Meta-Learning for Stochastic Gradient MCMC", the authors present a meta-learning approach to automatically design MCMC sampler based on Hamiltonian dynamics to mix faster on problems similar to the training problems. The approach is evaluated on simple multidimensional Gaussians, and Bayesian neural networks (including fully connected, convolutional, and recurrent networks).
MCMC samplers in general, and Hamiltonian Monte Carlo sampler in particular, are very powerful tools to perform Bayesian inference in high-dimensional spaces. Combined with stochastic gradients, methods like Stochastic Gradient MCMC (SGMCMC), or Stochastic Gradient Langevin Dynamics (SGLD) have been successfully used to apply these methods in the large data regime, where only noisy estimates of the gradients are feasible. Even though, many different samplers exists, and they are provably correct (meaning they converge to the correct distribution), fast mixing and low auto-correlation within the chain can heavily depend on the problem at hand and the hyperparameters of the sampler used.  The work presented here, uses the general framework for SG-MCMC samplers of Ma et al., parametrizes it with a neural network and learns its weights on representative training problems.
The paper is well written, although occasional minor mistakes and typos can be found.
It seems however, that the method is still quite laborious and some care needs to be taken to train the meta-sampler.
The overall narrative is easy to follow, but could benefit from more detail in certain parts. In general, I argue for acceptance of the paper, but have the following questions/comments:
- Below Eq. (7), an interpretation of the parametrizations Q_f and D_f is given. I greatly appreciate this, but the phrase 'Q_f is
responsible for the acceleration of \theta' is not really instructive. By definition, the change in \theta is mostly driven by the momentum p. Therefor, Q_f looks like an inverse mass (at least in the second line of (7)), but maybe that is not a very helpful analogy either.
- at the beginning of section 3.2, the term 'particles' is used. While I am fully aware of what that is supposed to mean, a reader less familiar with the topic could be confused, because there is no explanation of it.
- It is unclear to me how the stochastic estimate \tilde{U}(\theta) in equation (10) is computed exactly. Is it estimated using the current mini-batch at time t, or is it estimated using a 'holdout-test set'?
- I was wondering how the correlation between the chains due to thinning for the In-chain loss affects the results. The text, does not address this at all.
- The experiments are very thorough and I appreciate the comparison to the tuned baselines, but I am missing some details in the paper:
     (a) Did you tune the SGHMC method in Figure 2, as well? It is not mentioned in the text, and the sample path looks very volatile, which could indicate a poor combination of step length and noise.
     (b) How was the tuning of the base line methods performed?
     (c) Are the results in Figure 3 based on single runs, or do you show the mean over 10 independent runs (as in table 1).
- The insets in Figure 6 are helpful, but I think you could shrink the 'outer y axis' and have the inset in the top right corner instead. That way, the zoomed-out plot would show more details on its own.