Recent work using auxiliary prediction task classifiers to investigate the properties of LSTM representations has begun to shed light on why pretrained representations, like ELMo (Peters et al., 2018)  and CoVe (McCann et al., 2017), are so beneficial for neural language understanding models. We still, though, do not yet have a clear understanding of how the choice of pretraining objective affects the type of linguistic information that models learn. With this in mind, we compare four objectives-language modeling, translation, skip-thought, and autoencoding-on their ability to induce syntactic and part-of-speech information. We make a fair comparison between the tasks by holding constant the quantity and genre of the training data, as well as the LSTM architecture. We find that representations from language models consistently perform best on our syntactic auxiliary prediction tasks, even when trained on relatively small amounts of data. These results suggest that language modeling may be the best data-rich pretraining task for transfer learning applications requiring syntactic information. We also find that the representations from randomly-initialized, frozen LSTMs perform strikingly well on our syntactic auxiliary tasks, but this effect disappears when the amount of training data for the auxiliary tasks is reduced.

This paper tests various pretraining objectives (language modeling, machine translation, skip-thought, and autoencoding) on two syntactic tasks: POS tagging and CCG tagging. It finds that language modeling outperforms the other pretraining objectives; additionally, randomly-initializing an encoder achieves decent performance when given a large amount of labeled data for the tagging task. The experiments in this paper are very thorough and explained well. By controlling for pretraining data size, the authors are able to reasonably claim that language modeling is superior to translation as a syntactic transfer learning task. On the other hand, I have some concerns regarding the significance of the paper's contributions, and as such I am borderline on its acceptance.
comments:
- the experiments in the paper feel biased towards language modeling. Language modeling is the only token-level prediction task of the four objectives here, but both of the two downstream tasks are at the token level. It is perhaps unsurprising then that language modeling performs best; perhaps the authors could have considered some sentence-level downstream tasks as well to properly control for this? Or added some more word-level pretraining objectives?
- sort of relatedly, the authors do not provide any explanations as to *why* language modeling is a better pretraining objective than translation. What kinds of examples do the tagging models using LM pretraining get right that the translation models do not? Such an analysis could help provide more concrete insights into what kind of information each objective is encoding.
- the claim that LMs > translation is not a new finding. The authors cite Blevins et al, who find the same result on the task of dependency arc prediction. Similarly, the surprisingly good performance of random encoders was also found in Conneau et al., ACL 2018. As the main contribution of this paper seems to be a more controlled study of Blevins et al on different syntactic tasks, I don't think there is enough here for an ICLR submission.
- what is the effect of the specific dataset and architecture on the results? Here we just look at a couple translation datasets (all news data) and LSTM models. Do things change when we move to transformers or more diverse domains?


Local Binary Pattern Networks
Memory and computation efficient deep learning architectures are crucial to the continued proliferation of machine learning capabilities to new platforms and systems, especially, mobile sensing devices with ultra-small resource footprints. In this paper, we demonstrate such an advance for the well-studied character recognition problem. We use a strategy different from the existing literature by proposing local binary pattern networks or LBPNet that can learn and perform bit-wise operations in an end-to-end fashion. Binarization of operations in convolutional neural networks has shown promising results in reducing the model size and computing efficiency. Characters consist of some particularly structured strokes that are suitable for binary operations. LBPNet uses local binary comparisons and random projection in place of conventional convolution (or approximation of convolution) operations, providing important means to improve memory and speed efficiency that is particularly suited for small footprint devices and hardware accelerators. These operations can be implemented efficiently on different platforms including direct hardware implementation. LBPNet demonstrates its particular advantage on the character classification task where the content is composed of strokes. We applied LBPNet to benchmark datasets like MNIST, SVHN, DHCD, ICDAR, and Chars74K and observed encouraging results.

In this work, a neural network that uses local binary patterns instead of kernel convolutions is introduced. Using binary patterns has two advantages: a) it reduces the network definition to a set of binary patterns (which requires much less storage than the floating point descriptions of the kernel weights used in CNNs) and b) allows for fast implementations relying only on logical operations (particularly fast on dedicated hardware).
This work is mostly descriptive of a proposed technique with no particular theoretical performance guarantees, so its value hinges mostly on its practical performance on real data. In that sense, its evaluation is relatively limited, since only figures for MNIST and SVHN are provided.
A list of additional datasets is provided in Table 5, but only the performance metric is listed, which is meaningless if it is not accompanied with figures for size, latency and speedup. The only takeway about the additional datasets is that the proposed LBPNet can match or outperform a weak CNN baseline, but we don't know if the latter achieves state-of-the-art performance (previous figures of the baseline CNN suggest it doesn't) and we don't know if there's significant gain in speed or size.
Regarding MNIST and SVHN, which are tested in some more detail, again, we are interested in the performance-speed (or size) tradeoff, and it is unclear that the current proposal is superior. The baseline CNN does not achieve state of the art performance (particularly in SVHN, for which the state-of-the-art is 1.7% and the baseline CNN achieves 6.8%). For SVHN, BCNN has a much better performance-speed tradeoff than the baseline, since it is both faster and higher performance. Then, the proposed method, LBPNet, has much higher speed, but lower performance than BCNN. It is unclear how LBPNet's and BCNN's speeds would compare if we were to match their performances. For this reason, it is unclear to me that LBPNet is superior to BCNN on SVHN.
Also the numbers in boldface are confusing, aren't they just incorrect for both the Latency and Error in MNIST? Same for the Latency in SVHN.
The description of the approach is reasonably clear and clarifying diagrams are provided. The backpropagation section seems a bit superficial and could be improved. For instance, backpropagation is computed wrt the binary sampling points, as if these were continuous, but they have been defined as discrete before. The appendix contains a bit more detail, where it seems that backpropagation is alternated with rounding. It's not justified why this is a valid gradient descent algorithm.
Also how the scaling k of the tanh is set is not explained clearly. Do you mean that with more sampling points k should be larger to keep the outputs of the approximate comparison operator close to 0 and 1?
Minor:
What exactly in this method makes it specific to character recognition? Since you are trying to capture both high-level and low-level frequencies, it seems you'd be capturing all the relevant information. SVHN data are color images with objects (digits) in it, what is the reason that makes other objects not be detectable with this approach?
English errors are pervasive throughout the paper. A non-exhaustive list:
Fig 4.b: X2 should be Y2
particuarly
"to a binary digits"
"In most case"
"0.5 possibility"
"please refer to Sec .."
"FORWARD PROPATATION"


We introduce a probabilistic framework for quantifying the semantic similarity between two groups of embeddings. We formulate the task of semantic similarity as a model comparison task in which we contrast a generative model which jointly models two sentences versus one that does not. We illustrate how this framework can be used for the Semantic Textual Similarity tasks using clear assumptions about how the embeddings of words are generated. We apply information criteria based model comparison to overcome the shortcomings of Bayesian model comparison, whilst still penalising model complexity. We achieve competitive results by applying the proposed framework with an appropriate choice of likelihood on the STS datasets.

The authors propose a probabilistic model for computing the sentence similarity between two sets of representations in an online fashion (that is, they do not need to see the entire dataset at once as SIF does when using PCA). They evaluate on the STS tasks and outperform competitive baselines like WMD, averaging embeddings, and SIF (without PCA), but they have worse performance that SIF + PCA.
The paper is clearly written and their model is carefully laid out along with their derivation. My concern with this paper however, is that I feel the paper lacks a motivation, was it derive an online similarity metric that outperforms SIF(without PCA)?
A few experimental questions/comments:
What happens to all methods when stop words are not removed? How far does performance fall? I think one reason it might fall (in addition to the reasons given in the paper) is that all vectors are set to have the same norm. For STS tasks, often the norms of these vectors are reduced during training which lessens their influence. What mechanism was used to identify the stop words and does removing these help the other methods (I know in the paper, stop words were removed in the baseline, did this unilaterally improve performance for these methods)?
Overall I do like the paper, however I do find the results to be lackluster. There are many papers on combining word embeddings trained in various ways that have much stronger numbers on STS, but these methods won't be effective with this type of similarity (namely because embeddings must have unit norm in their model). Therefore, I think the paper needs some more motivation and experimental evidence of its superiority over related methods like SIF+PCA in order for it to be accepted.
PROS
- Probabilistic model with clear design assumptions from which a similarity metric can be derived.
- Derived similarity metric doesn't require knowledge of the entire dataset (in comparison to SIF + PCA)
CONS
- Performance seems to be slightly better than SIF, WMD, and averaging word embeddings, but below that of SIF + PCA
- Unclear motivation for the model, was it derive an online similarity metric that outperforms SIF(without PCA)?
- Requires the removal of stop words, but doesn't state how these were defined. Minor point, but tuning this could be enough to cause the improvement over related methods.


Deep reinforcement learning (deep RL) research has grown significantly in recent years. A number of software offerings now exist that provide stable, comprehensive implementations for benchmarking. At the same time, recent deep RL research has become more diverse in its goals. In this paper we introduce Dopamine, a new research framework for deep RL that aims to support some of that diversity. Dopamine is open-source, TensorFlow-based, and provides compact yet reliable implementations of some state-of-the-art deep RL agents. We complement this offering with a taxonomy of the different research objectives in deep RL research. While by no means exhaustive, our analysis highlights the heterogeneity of research in the field, and the value of frameworks such as ours.

Summary:
The authors present an open-source framework TensorFlow-based named Dopamine to facilitate the task of researchers in deep reinforcement learning (deep RL). It allows to build deep RL using existing components such as reinforcement learning agents, as well as handling memory, logs and providing checkpoints for them.
Emphasis is given on providing a unified interface to these agents as well as keeping the framework generic and simple (2000 lines of code).
The framework was demonstrated on Atari games notably using Deep Q-network agents (DQN).
The authors provide numerous examples of parameter files that can be used with their framework.
Performance results are reported for some agents (DQN, C51, Rainbow, IQN).
Given the actual trends in deep learning works, unified frameworks such as that proposed is welcome.
The automatization of checkpointing for instance is particularly useful for long running experiments.
Also, trying to reduce the volume of code is beneficial for long-term maintenance and usability.
Major concerns:
* This type of contribution may not match the scope of ICLR.
* In the abstract and a large fraction of the text, the authors claim that their work is a generic reinforcement learning framework. However, the paper shows that the framework is very dependent on agents playing Atari games. Moreover, the word "Atari" comes out of nowhere on pages 2 and 5.
The authors should mention in the beginning (e.g. in the abstract) that they are handling only agents operating on Atari games.
* The positioning of the paper relative to existing approaches is unclear: state of the art is mentioned but neither discussed nor compared to the proposal.
* The format of the paper should be revised:
                - Section 5 (Related Works) should come before presenting the author's work. When reading the preceding sections, we do not know what to expect from the proposed framework.
                - All the code, especially in the appendices, seems not useful in such a paper, but rather to the online documentation of the author's framework.
* What is the motivation of the author's experiments?
                - Reproduce existing results (claimed on page 1)? Then, use the same settings as published works and show that the author's framework reaches the same level of performances.
                - Show new results (such as the effect of stickiness)? Then the authors should explicitly say that one of the contributions of the paper is to show new results.
* The authors say that they want to compare results in Figure 3. They explain why the same scale is not used. To my opinion, the authors should find a way to bring all comparisons to the same scale.
For all these reasons, I think the paper is not ready for publication at ICLR.


Recently, Generative Adversarial Networks (GANs) have emerged as a popular alternative for modeling complex high dimensional distributions. Most of the existing works implicitly assume that the clean samples from the target distribution are easily available. However, in many applications, this assumption is violated. In this paper, we consider the problem of learning GANs under the observation setting when the samples from target distribution are given by the superposition of two structured components. We propose two novel frameworks: denoising-GAN and demixing-GAN. The denoising-GAN assumes access to clean samples from the second component and try to learn the other distribution, whereas demixing-GAN learns the distribution of the components at the same time. Through comprehensive numerical experiments, we demonstrate that proposed frameworks can generate clean samples from unknown distributions, and provide competitive performance in tasks such as denoising, demixing, and compressive sensing.

Quality is good, just a handful of typos.
Claritys above average in explaining the problem setting.
Originality: scan refs...
Significance: medium
Pros: the authors develop a novel GAN-based approach to denoising, demixing, and in the process train generators for the various components (not just inference). Further, for inference, the authors propose an explicit procedure. It seems like a noveel approach to demixing which is exciting.
Cons: The experiments do not push the limits of their method. It's difficult to judge the demixing 'power' of the method because it's difficult to tell how hard the problem is. Their method seems to easily solve it (super low MSE). The classification measure is clearly improved by denoising, which is totally unsurprising-- There should definitely be comparison with other denoising methods.
In general, they don't compare to any other methods. Actually in the appendix, comparisons are provided for a basic compressive sensing problem, but their only comparator is "LASSO" with a "fixed regularization parameter", and vanilla GAN. Since the authors "main contribution" (their words) is demixing, I'm surprised that they did not compare with other demixing approaches, or try on a harder problem. Could you  give some more details about the LASSO approach? How did you choose the L1 parameter?
I have another problem with the demixing experimental setting. On one hand, both the sinusoids and MNIST have "similar characteristics" in the sense that they are both pretty sparse, basically simple combinations of primary curves. This actually makes the problem harder for a dictionary learning approach like MCA (referenced in your paper). On the other hand, both signals are very simple to reconstruct. For example, what if you superimposed the grid of digits onto a natural image? Would you be able to train the higher resolution GAN to handle a more difficult setting? The other demixing setting of adding 1's and 2's has a similar problem.
The authors need to provide (R)MSE  results that show how well the method can reconstruct mixture components on average over the dataset. The only comparison is visual, and no comparators are provided.
Conclusions:
I'm actually torn on this paper. On one hand this paper seems novel and clearly contributes to the field. On the other hand, HOW MUCH contribution is not addressed experimentally, i.e. the method is not properly compared with other denoising or demixing methods, and definitely not pushed to its limits. It's hard to assess the difficulty of the denoising problem because their method does so well, and it's hard to assess the difficulty of demixing because of the lack of comparators.
Caveats:
I am knowledgeable about iterative optimization approaches to denoising and demixing, especially MCA (morphological component analysis), but *not knowledgeable about GAN-based approaches*, though I have familiarity with GANs.
*********************
Update after author response:
I think the Fashion-MNIST experiments and comparisons with ICA are many times more compelling than the original experiments. I think this is an exciting contribution to dually learning component manifolds for demixing.


We present SOSELETO (SOurce SELEction for Target Optimization), a new method for exploiting a source dataset to solve a classification problem on a target dataset. SOSELETO is based on the following simple intuition: some source examples are more informative than others for the target problem. To capture this intuition, source samples are each given weights; these weights are solved for jointly with the source and target classification problems via a bilevel optimization scheme. The target therefore gets to choose the source samples which are most informative for its own classification task. Furthermore, the bilevel nature of the optimization acts as a kind of regularization on the target, mitigating overfitting. SOSELETO may be applied to both classic transfer learning, as well as the problem of training on datasets with noisy labels; we show state of the art results on both of these problems.

PROS:
* This is an interesting approach of assigning contribution weights to each source sample.
* Could be very helpful for tasks where we have a noisy and a (small) clean dataset.
* The method seems to be performing well for the tasks chosen, especially for the CIFAR experiments.
* Simple idea and relatively easy to implement
CONS:
* Clarity could be improved, especially in the experimental section
* The motivation for the SVHN 0-4 to MNIST 5-9 is not clear. It would make more sense to me to transfer between SVHN 0-5 to SVHN 5-9, or from the entire SVHN to the entire MNIST, but this particular transfer seems somewhat irrelevant to the claims. The two domains are particularly dissimilar and trying to select "good" SVHN samples according to 20 or 25 MNIST samples seems somewhat ill-posed. It is also particularly surprising to me that 25 MNIST samples were enough to train a LeNet to the point of 84% accuracy on the entire MNIST test set. (I'm referring to the target-only line) Is that really the case, or was a larger training set used for that particular line?
* There is a claim that "SOSELETO has superior performance to all of the techniqiues which do not use unlabelled data", however I'm not sure whether these techniques were used as prescribed and if the comparison was fair. For example, I believe domain adaptation techniques like DANN, largely assume a common label space between the domains.
* Comparison with previous re-weighting techniques would have been very informative.
QUALITY:
* The quality of the writing was overall high, with a few exceptions, including the related work and the experimental section.
* In related work, the "bilevel optimization" section could be a bit more descriptive, maybe some of the explanationgiven in Sect. 3 could be moved here?
* The experiments were convincing, with the exception of the SVHN to MNIST section.
CLARITY:
* I believe a better synthetic experiment could be chosen to highlight the approach: how about a truly noisy dataset that is not as separable as the "noisy" dataset in Figure 1? Maybe you could have the same noisy dataset but with a small portion of random points having the wrong label. For the same experiment, it should be clearly stated that your task is binary classification and what was the classifier used.
* For the CIFAR experiments, it is very good that it performs well, but it'd be informative to see if SOSELETO can perform even better with 10K samples.
* It wasn't clear to me whether the a-values of only one batch (32 samples?) at a time were affected. If so, how does this scale to really large datasets like, say, Imagenet?
* In the CIFAR experiments, it is mentioned that a target batch-size is chosen to be larger to enable more a-values to be affected. This seems like a typo, but it was confusing. (I assume that the source batch-size is chosen to be larger)
* Figure 2 could use a better caption and a legend. It would also be an easier figure to parse if the x-axis was reversed (eg. if the x-axis was the fraction of data used)
* It was not clear to me what "true transfer learning" means as opposed to domain adaptation.
ORIGINALITY:
* It seems that this idea has been explored before, however I'm not personally familiar with that work. I would have definitely liked to see comparisons with it though.
SIGNIFICANCE:
* This is a simple idea that seems to work well. As I wrote above, it would be great to know how it compares to other re-weighting techniques.


MINIMAL RANDOM CODE LEARNING: GETTING BITS BACK FROM COMPRESSED MODEL PARAMETERS
While deep neural networks are a highly successful model class, their large memory footprint puts considerable strain on energy consumption, communication bandwidth, and storage requirements. Consequently, model size reduction has become an utmost goal in deep learning. A typical approach is to train a set of deterministic weights, while applying certain techniques such as pruning and quantization, in order that the empirical weight distribution becomes amenable to Shannon-style coding schemes. However, as shown in this paper, relaxing weight determinism and using a full variational distribution over weights allows for more efficient coding schemes and consequently higher compression rates. In particular, following the classical bits-back argument, we encode the network weights using a random sample, requiring only a number of bits corresponding to the KullbackLeibler divergence between the sampled variational distribution and the encoding distribution. By imposing a constraint on the Kullback-Leibler divergence, we are able to explicitly control the compression rate, while optimizing the expected loss on the training set. The employed encoding scheme can be shown to be close to the optimal information-theoretical lower bound, with respect to the employed variational family. Our method sets new state-of-the-art in neural network compression, as it strictly dominates previous approaches in a Pareto sense: On the benchmarks LeNet-5/MNIST and VGG-16/CIFAR-10, our approach yields the best test performance for a fixed memory budget, and vice versa, it achieves the highest compression rates for a fixed test performance.

The authors come up with a surprisingly elegant algorithm ("minimal random coding") which encodes samples from a posterior distribution, only using a number of bits that approximates the KL divergence between posterior and prior, while Shannon-type algorithms can only do this if the posterior is deterministic (a delta distribution). It can also be directly used to sample from continuous distributions, while Shannon-type algorithms require quantization. In my opinion, this is the main contribution of the paper.
The other part of the paper that is specifically concerned with weight compression ("MIRACLE") turns out to be a lot less elegant. It is somewhat ironic that the authors specifically call attention to the their algorithm sending random samples, as opposed to existing algorithms, which quantize and then send deterministic variables. This is clearly true for the basic algorithm, but, if I understand correctly, not for MIRACLE. It seems clear that neural networks are sensitive to random resampling of their weights -- otherwise, the authors would not have to fix the weights in each block and then do further gradient descent for the following blocks. What would happen if the distributions were held constant, and the algorithm would be run again, just with a different (but identical) random seed in both sender and receiver? It seems this would lead to a performance drop, demonstrating that (albeit indirectly), MIRACLE also makes a deterministic choice of weights.
Overall, I find the paper somewhat lacking in terms of evaluation. MIRACLE consists of a lot of parts. It is hard to assess how much of the final coding gain presented in table 1 is due to the basic algorithm. What is the effect of selecting other probability models, possibly different ones than Gaussians? Choosing appropriate distributions can have a significant impact on the value of the KL divergence. Exactly how much is gained by applying the hashing trick? Are the standard deviations of the priors included in the size, and how are they encoded?
This could be assessed more clearly by directly evaluating the basic algorithm. Theorem 3.2 predicts that the approximation error of algorithm 1 asymptotically zero, i.e. one can gain an arbitrarily good approximation to the posterior by spending more bits. But how many more are practically necessary? It would be fantastic to actually see some empirical data quantifying how large the error is for different distributions (even simple toy distributions). What are the worst vs. best cases?


When a bilingual student learns to solve word problems in math, we expect the student to be able to solve these problem in both languages the student is fluent in, even if the math lessons were only taught in one language. However, current representations in machine learning are language dependent. In this work, we present a method to decouple the language from the problem by learning language agnostic representations and therefore allowing training a model in one language and applying to a different one in a zero shot fashion. We learn these representations by taking inspiration from linguistics, specifically the Universal Grammar hypothesis and learn universal latent representations that are language agnostic (Chomsky, 2014;Montague, 1970). We demonstrate the capabilities of these representations by showing that models trained on a single language using language agnostic representations achieve very similar accuracies in other languages.

This paper starts with the bold aim of extracting Montague's universal grammar from multiple languages. In order to do so, the authors train multiple language models where each LM is explicitly factorized into language-specific and language-independent representations. The authors then apply the GAN framework to the language-independent parts to enforce all languages to share the same latent space. The claim here is that the language independent parameters capture the essence of universal grammar. The authors show that their framework enables effective zero-shot learning of tasks over new languages (for example sentiment classifier learned on top of English data generalize to Chinese when trained on the universal grammar embeddings).
The paper is overall well written and the experimental results are convincing.
The gripe, however, I have is that this paper makes the claims that go too far without evaluating them. It is entirely sufficient to claim that you're trying to learn language agnostic parameters/embeddings --  I'd be happy with that. But the paper goes further and claims to be learning a form of universal grammar. To justify this claims, it is not sufficient to show in the experiments that the new representations do better at sentiment and NLI. The authors must show that this captures the "innate" language learning abilities akin to human babies. While the paper aims to do some analysis in the discussion section, it is not unsatisfactory. As the paper says in the discussion section "From a machine learning perspective, we’re interested in extracting informative features and not necessarily a completely grammatical language model. That being said it is of interest to what extent language models capture grammar and furthermore the extent to which models trained toward the universal grammar objective learn grammar."
The problem is that simply comparing LM perplexities is not a solid test of whether this model has learned some form of universal grammar. First, this paper does not define a clear falsifiable hypothesis on the proof of learning universal grammar. One example of testing for learning grammar can be: does this model learn basic syntactic rules of a new language (e.g. as the authors suggested -- head-first or head-final syntax, or rules of conjugation)  with a small amount of data after being training a universal representation with n languages? There have been a series of recent papers on checking if Language Models have appropriately learned syntax. See e.g. Tal Linzen's work https://arxiv.org/pdf/1809.04179.pdf. Just to be clear I am not suggesting citing works in unpublished places but potentially using some of the tests suggested in these papers.
In conclusion, I think this work is useful but I also think it makes really grandiloquent claims without verifying them. That to me is a dangerous precedent.


Neural Machine Translation (NMT) systems rely on large amounts of parallel data. This is a major challenge for low-resource languages. Building on recent work on unsupervised and semi-supervised methods, we present an approach that combines zero-shot and dual learning. The latter relies on reinforcement learning, to exploit the duality of the machine translation task, and requires only monolingual data for the target language pair. Experiments on the UN corpus show that a zero-shot dual system, trained on English-French and English-Spanish, outperforms by large margins a standard NMT system in zero-shot translation performance on SpanishFrench (both directions). We also evaluate on newstest2014. These experiments show that the zero-shot dual method outperforms the LSTM-based unsupervised NMT system proposed in (Lample et al., 2018b), on the en− →fr task, while on the fr− →en task it outperforms both the LSTM-based and the Transformers-based unsupervised NMT systems.

Pros:
- The paper address the problem of zero-shot translation. The proposed method is essentially to bootstrap a Dual Learning process using a multilingual translation model that already has some degree of zero-shot translation capabilities. The idea is simple, but the approach improves the zero-shot translation performance of the baseline model, and seems to be better than either pivoting or training on direct but out-of-domain parallel data.
- The paper is mostly well written and easy to follow. There are some missing details that I've listed below.
Cons:
- There is very little comparison to related work. For example, related work by Chen et al. [1], Gu et al. [2] and Lu et al. [3] are not cited nor compared against.
Misc questions/comments:
- In a few places you call your approach unsupervised (e.g., in Section 3: "Our method for unsupervised machine translation works as follows: (...)"; Section 5.2 is named "Unsupervised Performance"). But your method is not unsupervised in the traditional sense, since you require lots of parallel data for the target languages, just not necessarily directly between the pair. This may be unrealistic in low-resource settings if there is not an existing suitable pivot language. It'd be more accurate to simply say "zero-shot" (or maybe "semi-supervised") in Section 3 and Section 5.2.
- In Section 3.1 you say that your process implements the three principles outlined in Lample et al. (2018b). However, the Initialization principle in that work refers to initializing the embeddings -- do you pretrain the word embeddings as well?
- In Section 4 you say that the "UN corpus is of sufficient size". Please mention what the size is.
- In Section 4.2, you mention that you set dropout to p=0.65 when training your language model -- this is very high! Did you tune this? Does your language model overfit very badly with lower dropout values?
- In Section 5.2, what is the BLEU of an NMT system trained on the es->fr data (i.e., what is the upper bound)? What is the performance of a pivoting model?
- In Section 5.3, you say you use "WMT News Crawl, all years." Please indicate which years explicitly.
- In Table 3, what is the performance of a supervised NMT system trained on 1M en-fr sentences of the NC data? Knowing that would help clarify the impact of the domain mismatch.
- minor comment: in Section 4.3 you say that you trained on Tesla-P100, but do you mean Pascal P100 or Tesla V100?
[1] Chen et al.: http://aclweb.org/anthology/P17-1176
[2] Gu et al.: http://aclweb.org/anthology/N18-1032
[3] Lu et al.: http://www.statmt.org/wmt18/pdf/WMT009.pdf


Saliency map, or the gradient of the score function with respect to the input, is the most basic means of interpreting deep neural network decisions. However, saliency maps are often visually noisy. Although several hypotheses were proposed to account for this phenomenon, there is no work that provides a rigorous analysis of noisy saliency maps. This may be a problem as numerous advanced attribution methods were proposed under the assumption that the existing hypotheses are true. In this paper, we identify the cause of noisy saliency maps. Then, we propose Rectified Gradient, a simple method that significantly improves saliency maps by alleviating that cause. Experiments showed effectiveness of our method and its superiority to other attribution methods. Codes and examples for the experiments will be released in public.

In the paper, the authors proposed a new saliency map method, based on some empirical observations about the cause of noisy gradients.
Specifically, through experiments, the authors clarified that the noisy gradients are due to irrelevant information propagated in the forward pass in DNN. Because the backpropagation follows the same pass, irrelevant feature are conveyed back to the input, which results in noisy gradients.
To avoid noisy gradients, the authors proposed a new backpropagation named Rectified Gradient (RectGrad). In RectGrad, the backward pass is filtered out if the product of the forward signal and the backward signal are smaller than a threshold. The authors claim that, with this modification in backpropagation, the gradients get less noisy.
In some experiments, the authors presented that RectGrad can produce clear saliency maps.
I liked the first half of the paper: the observations that irrelevant forward passes are causing noisy gradients seem to be convincing. The experiments are designed well to support the claim.
Here, I would like to point out, that noisy gradients in occluded images may be because of the convolutional structures. Each filter in convolution layer is trained to respond to certain patterns. Because the same filter is used for each of subimages, some filters can be activated occasionally on occluded parts. I think this does not happen if the network is densely connected without convolutional structures. The trained dense connection will be optimized to remove the effects of occluded parts. Hence, for such networks, the gradient will be zeros for occluded parts.
The second half of the paper (Sec.4 and 5) are not very much convincing to me.
Below, I raise several concerns.
1. There is no justification on the definition of RectGrad: Why Rl = I(al * Rl > t) R(l+1)?
The authors presented Rl = I(al * Rl > t) R(l+1) as RectGrad, that can filter out irrelevant passes. However, there is no clear derivation of this formula: the definition suddenly appears. If the irrelevant forward passes are causes of noisy gradients, the modification Rl = I(al > t) R(l+1) seems to be more natural to me. It is also a natural extension to the ReLU backward pass Rl = I(al > 0) R(l+1). Why we need to filter out negative signals in backward pass?
2. The experimental results are less convincing: Is RectGrad truly good?
In Sec.5.2, the authors presented saliency maps only on a few images, and claimed that they look nicely. However, it is not clear that those "nicely looking" saliency map are truly good ones. I expect the authors to put much efforts on quantitative comparisons rather than qualitative comparisons, so that we can understand that those "nicely looking" saliency maps are truly good ones.
Sec.5.3 presents some quantitative comparisons, however, the reported Sensitivity and ROAR/KAR on RectGrad are not significant. The authors mentioned that this may be because of the sparsity of RectGrad. However, if the sparsity is the harm, the underlying observations of RectGrad may have some errors. I think the current manuscript has an inconsistency between the fundamental idea (based on empirical observations) and the performance of RectGrad.
[Minor Concern]
In Sec.5, the authors frequently refer to the figures in appendix. I think the main body of the paper should be self-contatined. I therefore think that some of the figures related to main results should appear in the main part.


Generative Adversarial Networks (GANs) are an elegant mechanism for data generation. However, a key challenge when using GANs is how to best measure their ability to generate realistic data. In this paper, we demonstrate that an intrinsic dimensional characterization of the data space learned by a GAN model leads to an effective evaluation metric for GAN quality. In particular, we propose a new evaluation measure, CrossLID, that assesses the local intrinsic dimensionality (LID) of input data with respect to neighborhoods within GAN-generated samples. In experiments on 3 benchmark image datasets, we compare our proposed measure to several state-of-the-art evaluation metrics. Our experiments show that CrossLID is strongly correlated with sample quality, is sensitive to mode collapse, is robust to small-scale noise and image transformations, and can be applied in a modelfree manner. Furthermore, we show how CrossLID can be used within the GAN training process to improve generation quality.

Statistics based on KNN distances are ubiquitous in machine learning. In this paper the authors propose to apply the existing LID metric to GANs. The metric can be decomposed as follows: (1) Given a point x in X, compute the k-nearest neighbors KNN(x, X) and let those distances be R1, R2, …, Rk. Now, rewrite LID(x, X) = [max_over_i (log Ri) - mean_over_i (log Ri)] to uncover that the distribution of (log-)distances is summarized as a function of the max distance and the mean distance. (2) To extend the metric to two sets, A and B, define CrossLID(A; B) = E_(x in A) [LID(x, B)]. To see why CrossLID is useful, let X be the observed data and G the generated data. First consider CrossLID(A, B) where A=B=X which determines a lower-bound which is essentially the average (over elements of A) LID statistic determined by the underlying KNN graph of X. Now, keep A=X, and progressively change B to G (say by replacing some points from X with some points from G). This will induce a change of the distance statistics of some points from A, which will be detected on the individual LID scores of those points, and will hence be propagated to CrossLID. As a result, LID close to the baseline LID detects both sample quality issues as well as mode dropping/collapse issues. In practice, instead of computing this measure in the pixel space, one can compute it in the feature space of some feature extractor, or in some cases directly in the learned feature space of the generator. Finally, given some labeling of the points, one can keep track of the CrossLID statistic for each mode and use this during training to oversample modes for which the gap between the expected CrossLID and computed one is large.
Clarity: I think the clarity can be improved -- instead of stating the (rather abstract) properties of LID, the readers might benefit from the direct discussion of the LID estimator and a couple of examples, derive the max - mean relationship for the MLE estimator and provide some guiding comments. In a later section one might discuss why the estimator is so powerful and generally applicable. Secondly, the story starts with “discriminability of the distance measure” and the number of latent variables needed to do it, but I felt that this only complicated matters as many of these concepts are unclear at this point.
Originality: Up to my knowledge, the proposed application is novel, albeit built on an existing (well-known) estimator. Nevertheless, the authors have demonstrated several desirable properties which might be proven useful in practice.
Significance of this work: The work is timely and attempts to address a critical research problem which hinders future research on deep generative models.
Pro:
- Generally well written paper, although the clarity of exposition can be improved.
- Estimator is relatively easy to compute in practice (i.e. the bottleneck will still be in the forward and backward passes of the DNNs).
- Can be exploited further when labeled data is available
- Builds upon a strong line of research in KNN based estimators.
- Solid experimental setup with many ablation studies.
Con:
- FID vs CrossLID: I feel that many arguments against FID are too strong. In particular, in “robustness to small input noise” and “robustness to input transformation” you are changing the underlying distribution *significantly* -- why should the score be invariant to this? After all, it does try to capture the shift in distribution. In the robustness to sample size again FID is criticized to have a high-variance in low-sample size regime: This is well known, and that’s why virtually all work presenting FID results apply 10k samples and average out the results over random samples. In this regime it was observed that it has a high bias and low variance (Figure 1 in [1]). In terms of the dependency of the scores to an external model, why wouldn’t one compute FID on the discriminator feature space? Similarly, why wouldn’t one compute FID in the pixel space and get an (equally bad) score as LID in pixel space? Given these issues, in my opinion, Table 1 overstates the concerns with FID, and understates the issues with CrossLID.
- FID vs CrossLID in practice: I argue that the usefulness comes from the fact that relative model comparison is sound. From this perspective it is critical to show that the Spearman’s rank correlation between these two competing approaches on real data sets is not very high -- hence, there are either sample quality or mode dropping/collapsing issues detected by one vs the other. Now, Figure 1 in [1] shows that this FID is sensitive to mode dropping. Furthermore, FID is also highly correlated with sample quality (page 7 of [2]).
- A critical aspect here is that in pixel space of large dimension the distances will tend to be very similar, and hence all estimators will be practically useless. As such, learning the proper features space is of paramount importance. In this work the authors suggest two remedies: (1) Compute a feature extractor by solving a surrogate task and have one extractor per data set. (2) During the training of the GAN, the discriminator is “learning” a good feature space in which the samples can be discriminated. Both of these have significant drawbacks. For (1) we need to share a dataset-specific model with the community. This is likely to depend on the preprocessing, model capacity, training issues, etc.. Then, the community has to agree to use one of these. On the other hand, (2) is only useful for biasing a specific training run. Hence, this critical aspect is not addressed and the proposed solution, while sensible, is unlikely to be adopted.
- Main contributions section is too strong -- avoiding mode collapse was not demonstrated. Arguably, given labeled data, the issue can be somewhat reduced if the modes correspond to labels. Similarly, if the data is well-clusterable one can expect a reduction of this effect. However, as both the underlying metric as well as the clustering depends on the feature space, I believe the claim to be too strong. Finally, if we indeed have labels or some assumptions on the data distribution, competing approaches might exploit it as well (as done with i.e. conditional GANs).
- In nonparametric KNN based density estimation, one often uses statistics based on KNN distances. What is the relation to LID?
With respect to the negative points above, without having a clear cut case why this measure outperforms and should replace FID in practice, I cannot recommend acceptance as introducing yet another measure might slow down the progress. To make a stronger case I suggest:
(1) Compute Spearman's rank correlation between FIDs and CrossLIDs of several trained models across these data sets.
(2) Compute the Pearson's correlation coefficient across the data sets. Given that your method has access to dataset specific feature extractors I expect it perform significantly better than FID.

[1] https://arxiv.org/pdf/1711.10337.pdf
[2] https://arxiv.org/pdf/1806.00035.pdf
========
Thank you for the detailed responses. I have updated my score from 5 to 6.


LEARNING TO LEARN WITH CONDITIONAL CLASS DEPENDENCIES
Neural networks can learn to extract statistical properties from data, but they seldom make use of structured information from the label space to help representation learning. For example "cat" and "dog" are closer than "cat" and "truck". Although some label structure can implicitly be obtained when training on huge amounts of data, in a few-shot learning context where little data is available, making explicit use of the label structure can inform the model to reshape the representation space to reflect a global sense of class dependencies. We propose a meta-learning framework, Conditional class-Aware Meta-Learning (CAML), that conditionally transforms feature representations based on a metric space that is trained to capture inter-class dependencies. This enables a conditional modulation of the feature representations of the base-learner to impose regularities informed by the label space. Experiments show that the conditional transformation in CAML leads to more disentangled representations and achieves competitive results on the miniImageNet benchmark.

[Summary]
The paper presents an enhancement to the Model-Agnostic Meta-Learning (MAML) framework to integrate class dependency into the gradient-based meta-learning procedure. Specifically, the class dependency is encoded by embedding the training examples via a clustering network into a metric space where semantic similarity is preserved via affinity under Euclidean distance. Embedding of an example in this space is further employed to modulate (scale and shift) features of the example extracted by the base-learner via a transformation network, and the final prediction is made on top of the modulated features. Experiments on min-ImageNet shows that the proposed approach improves the baseline of MAML.
Pros
- An interesting idea of leveraging class dependency in meta-learning.
- Solid implementation with reasonable technical solutions.
Cons
- Some relevant interesting areas/cases were not exploited/tested.
- Improvement over state-of-the-arts (SOA) is marginal or none.
[Originality]
The paper is motivated by an interesting observation that class dependency in the label space can also provide insights for meta-learning. This seems to be first introduced in the context of  meta-learning.
[Quality]
Overall the paper is well executed in some aspects, including motivation and technical implementation. There are, however, a few areas/cases I would like to see more from it so as to make a stronger case.
In terms of generalization, the proposed enhancement to MAML is claimed to be orthogonal to other SOAs that are also within the framework based on gradient-descent, e.g. LEO. It is not quite clear to me that if the use of class dependency can lead to general benefits to alike methods like LEO, or if it is just a specific case for the MAML baseline. Actually, it would be interesting to see how the proposed class-conditional modulation can help other SOA in table 1. Also, more empirical results from other use cases (e.g., other datasets or problems) also help provide more insights here. These augmentation can better justify the value or significance of this work.
In the specific formulation of the approach in Fig 2, it looks to me that the whole system is a compounded framework that combines two classifiers with one (base-learner) producing base representation, and the second injects side-information (e.g., from class-dependency in this case) to modulates the base representation before the final prediction. I just wonder what would happen if similar process keeps on? E.g., by building the third stage that modulates the features from the previous two? Or what if we swap the roles of base-learner and the embedding from the metric space (i.e., using the base-learner to modulate the embedding)? It looks to me that the feature/embedding from both components (in Fig 5 and 6) are optimized to improve separability. The roles they play in this process are also very interesting to get more elucidation.

Another point worth discussion is that the class dependency currently imposed does not see to include hierarchical structure among classes, i.e., the label space is still flat. It would be great if this can be briefly discussed with respect to the current formulation to better inspire the future work.
[Clarity]
The paper is generally well written and I did not have much difficulty to follow.
[Significance]
While the paper is built on an interesting idea, there are still a few areas for further improvement to justify its significance (the the comments above).


HARMONIZING MAXIMUM LIKELIHOOD WITH GANS FOR MULTIMODAL CONDITIONAL GENERATION
Recent advances in conditional image generation tasks, such as image-to-image translation and image inpainting, are largely accounted to the success of conditional GAN models, which are often optimized by the joint use of the GAN loss with the reconstruction loss. However, we reveal that this training recipe shared by almost all existing methods causes one critical side effect: lack of diversity in output samples. In order to accomplish both training stability and multimodal output generation, we propose novel training schemes with a new set of losses named moment reconstruction losses that simply replace the reconstruction loss. We show that our approach is applicable to any conditional generation tasks by performing thorough experiments on image-to-image translation, super-resolution and image inpainting using Cityscapes and CelebA dataset. Quantitative evaluations also confirm that our methods achieve a great diversity in outputs while retaining or even improving the visual fidelity of generated samples.

The paper proposes a modification to the traditional conditional GAN objective (which minimizes GAN loss as well as either L1 or L2 pixel-wise reconstruction losses) in order to promote diverse, multimodal generation of images. The modification involves replacing the L1/L2 reconstruction loss -- which predicts the first moment of a pixel-wise gaussian/laplace (respectively) likelihood model assuming a constant spherical covariance matrix -- with a new objective that matches the first and second moments of a pixel-wise gaussian/laplace likelihood model with diagonal covariance matrix. Two models are proposed for matching the first and second moments - the first one involves using a separate network to predict the moments from data which are then used to match the generator’s empirical estimates of the moments (using K samples of generated images). The second involves directly matching the empirical moment estimates using monte carlo.
The paper makes use of a well-established idea - modeling pixel-wise image likelihood with a diagonal covariance matrix i.e. heteroscedastic variance (which, as explained in [1], is a way to learn data-dependent aleatoric uncertainty). Following [1], the usage of first and second moment prediction is also prevalent in recent deep generative models (for example, [2]) i.e. image likelihood models predict the per-pixel mean and variance in the L2 likelihood case, for optimizing Equation 4 from the paper. Recent work has also attempted to go beyond the assumption of a diagonal covariance matrix (for example, in [3] a band-diagonal covariance matrix is estimated). Hence, the only novel idea in the paper seems to be the method for matching the empirical estimates of the first and second moments over K samples. The motivation for doing this makes intuitive sense since diversity in generation is desired, which is also demonstrated in the results.
Section specific comments:
- The loss of modality of reconstruction loss (section 3.2) seems like something which doesn’t require the extent of mathematical and empirical detail presented in the paper. Several of the cited works already mention the pitfalls of using reconstruction loss.
- The analyses in section 4.4 are sound in derivation but not so much in the conclusions drawn. It is not clear that the lack of existence of a generator that is an optimal solution to the GAN and L2 loss (individually) implies that any learnt generator using GAN + L2 loss is suboptimal. More explanation on this part would help.
The paper is well written, presents a simple idea, complete with experiments for comparing diversity with competing methods. Some theoretical analyses do no directly support the proposition - e.g. sections 3.2 and 4.4 in my specific comments above. Hence, the claim that the proposed method prevents mode collapse (training stability) and gives diverse multi-modal predictions is supported by experiments and intuition for the method, but not so much theoretically. However, the major weakness of the paper is the lack of novelty of the core idea.
=== Update after rebuttal:
Having read through the other reviews and the author's rebuttal, I am unsatisfied with the rebuttal and I do not recommend accepting the paper. My rating has decreased accordingly.
The reasons for my recommendation, after discussion with other reviews, are -- (1) lack of novelty and (2) weak theoretical results (some justification of which was stated in my initial review above). Elaborating more on the second point, I would like to mention some points which came up during the discussion with other reviewers: The theoretical result which states that not using reconstruction loss given that multi-modal outputs are desired is a weaker result than proving that the proposed method is actually effective in what it is designed to do. There are empirical results to back that claim, but I strongly believe that the theoretical results fall short and feel out of place in the overall justification for the proposed method. This, along with my earlier point of lack of novelty are the basis for my decision.
References:
[1] Kendall, Alex, and Yarin Gal. "What uncertainties do we need in bayesian deep learning for computer vision?." Advances in neural information processing systems. 2017.
[2] Bloesch, M., Czarnowski, J., Clark, R., Leutenegger, S., & Davison, A. J. (2018). CodeSLAM-Learning a Compact, Optimisable Representation for Dense Visual SLAM. CVPR 2018.
[3] Dorta, G., Vicente, S., Agapito, L., Campbell, N. D., & Simpson, I. (2018, February). Structured Uncertainty Prediction Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.


BEYOND GREEDY RANKING: SLATE OPTIMIZATION VIA LIST-CVAE
The conventional solution to the recommendation problem greedily ranks individual document candidates by prediction scores. However, this method fails to optimize the slate as a whole, and hence, often struggles to capture biases caused by the page layout and document interdepedencies. The slate recommendation problem aims to directly find the optimally ordered subset of documents (i.e. slates) that best serve users' interests. Solving this problem is hard due to the combinatorial explosion in all combinations of document candidates and their display positions on the page. Therefore we propose a paradigm shift from the traditional viewpoint of solving a ranking problem to a direct slate generation framework. In this paper, we introduce List Conditional Variational Auto-Encoders (List-CVAE), which learns the joint distribution of documents on the slate conditioned on user responses, and directly generates full slates. Experiments on simulated and real-world data show that List-CVAE outperforms popular comparable ranking methods consistently on various scales of documents corpora.

The latest revision is a substantially improved version of the paper. The comment about generalization still feels unsatisfying ("our model requires choosing c* in the support of P(c) seen during training") but could spur follow-up work attempting a precise characterization.
I remain wary of using a neural net reward function in the simulated environment, and prefer a direct simulation of Eqn5. With a non-transparent metric, it is much harder to diagnose whether the observed improvement in List-CVAE indeed corresponds to improved user engagement; or whether the slate recommender has gamed the neural reward function. Transparent metrics (that encourage non-greedy scoring) also have user studies showing they correlate with user engagement in some scenarios.
In summary, I think the paper is borderline leaning towards accept -- there is a novel idea here for framing slate recommendation problems in an auto-encoder framework that can spur some follow-up works.
---
The paper proposes using a variational auto-encoder to learn how to map a user context, and a desired user response to a slate of item recommendations. During training, data collected from an existing recommender policy (user contexts, displayed slate, recorded user response) can be used to train the encoder and decoder of the auto-encoder to map from contexts to a latent variable and decode the latent variable to a slate. Once trained, we can invoke the encoder with a new user context and the desired user response, and decode the resulting latent variable to construct an optimal slate.
A basic question for such an approach is: [Fig2] Why do we expect generalization from the user responses c(r) seen in training to c(r*) that we construct for testing? At an extreme, suppose our slate recommendation policy always picks the same k items and never gets a click. We can optimize Eqn3 very well on any dataset collected from our policy; but I don't expect deploying the VAE to production with c(r*) as the desired user response will give us anything meaningful.
The generalization test on RecSys2015-medium (Fig6d) confirms this intuition. Under what conditions can we hope for reliable generalization?
The comment about ranking evaluation metrics being unsuitable (because they favor greedy approaches) needs to be justified. There are several metrics that favor diversity (e.g. BPR, ERR) where a pointwise greedy scoring function will perform very sub-optimally.  Such metrics are more transparent than a neural network trained to predict Eqn6. See comment above for why I don't expect the neural net trained to predict Eqn6 on training data will not necessarily generalize to the testing regime we care about (at the core, finding a slate recommendation policy is asking how best to change the distribution P(s), which introduces a distribution shift between training and test regimes for this neural net).
There are 2 central claims in the paper: that this approach can scale to many more candidate items (and hence, we don't need candidate generation followed by a ranking step); and that this approach can reason about interaction-effects within a slate to go beyond greedy scoring. For the second claim, there are many other approaches that go beyond greedy (one of the most recent is Ai et al, SIGIR2018 https://arxiv.org/pdf/1804.05936.pdf; the references there should point to a long history of beyond-greedy scoring) These approaches should also be benchmarked in the synthetic and semi-synthetic experiments. At a glance, many neural rankers (DSSM-based approaches) use a nearly identical decoder to CVAE (one of the most recent is Zamani et al, CIKM2018 https://dl.acm.org/citation.cfm?id=3271800; the references there should point to many other neural rankers) These approaches should also be benchmarked in the expts.
This way, we have a more representative picture of the gain of CVAE from a more flexible (slate-level) encoder-decoder; and the gain from using item embeddings to achieve scalability.


KERNEL CHANGE-POINT DETECTION WITH AUXIL- IARY DEEP GENERATIVE MODELS
Detecting the emergence of abrupt property changes in time series is a challenging problem. Kernel two-sample test has been studied for this task which makes fewer assumptions on the distributions than traditional parametric approaches. However, selecting kernels is non-trivial in practice. Although kernel selection for two-sample test has been studied, the insufficient samples in change point detection problem hinders the success of those developed kernel selection algorithms. In this paper, we propose KL-CPD, a novel kernel learning framework for time series CPD that optimizes a lower bound of test power via an auxiliary generative model. With deep kernel parameterization, KL-CPD endows kernel twosample test with the data-driven kernel to detect different types of change-points in real-world applications. The proposed approach significantly outperformed other state-of-the-art methods in our comparative evaluation of benchmark datasets and simulation studies.

The manuscript entitled "Kernel Change-Point Detection with Auxiliary Deep Generative Models" describes a novel approach to optimising the choice of kernel towards increased testing power in this challenging machine learning problem.  The proposed method is shown to offer improvements over alternatives on a set of real data problems and the minimax objective identified is well motivated, however, I am not entirely convinced that (a) the performance improvements arise for the hypothesised reasons, and (b) that the test setting is of wide applicability.
A fundamental distinction between parametric and non-parametric tests for CPD in timeseries data is that the adoption of parametric assumptions allows for an easier introduction of strict but meaningful relationships in the temporal structure---e.g. a first order autoregressive model introduces a simple Markov structure---whereas non-parametric kernel tests typically imagine samples to be iid (before and after the change-point).  For this reason, the non-parametric tests may lack robustness to certain realistic types of temporal distributional changes: e.g. in the parameter of an autoregressive timeseries.  On the other hand, it may be prohibitively difficult to design parametric models to well characterise high dimensional data, whereas non-parametric models can typically do well in high dimension when the available data volumes are large.  In the present application it seems that the setting imagined is for low dimensional data of limited size in which there is likely to be non-iid temporal structure (i.e., outside the easy relative advantage of non-parametric methods).  For this reason it seems to me the key advantage offered by the proposed approach with its use of a distributional autoregressive process for the surrogate model may well be to introduce robustness against Type 1 errors due to otherwise unrepresented temporal structure in the base distribution (P).  In summarising the performance results by AUC it is unclear whether it is indeed the desired improvement in test power that offers the advantages or whether it is in fact a decrease in Type 1 errors.
Another side of my concern here is that I disagree with the statement: "As no prior knowledge of Q ... intuitiviely, we have to make G as close to P as possible" interpretted as a way to maximise test power; as a way to minimise Type 1 errors, yes.
Across change-point detection methods it is also important to distinguish key aspects of the problem formulation.  One particular specification here is that we have already some labelled instances of data known to come from the P distribution, and perhaps also some fewer instances of data labelled from Q.  This is distinct from fully automated change point detection methods for time series such as automatic scene selection in video data.  Another dissimilarity to that archetypal scenario is that here we suppose the P and Q distributions may have subtle differences that we're interested in; and it would also seem that we assume there is only one change-point to detect.  Or at least the algorithm does not seem to be designed to be applied in a recursive sense as it would be for scene selection.
Finally there is no discussion here of computational complexity and cost?


We present a probabilistic framework for studying adversarial attacks on discrete data. Based on this framework, we derive a perturbation-based method, Greedy Attack, and a scalable learning-based method, Gumbel Attack, that illustrate various tradeoffs in the design of attacks. We demonstrate the effectiveness of these methods using both quantitative metrics and human evaluation on various stateof-the-art models for text classification, including a word-based CNN, a characterbased CNN and an LSTM. As an example of our results, we show that the accuracy of character-based convolutional networks drops to the level of random selection by modifying only five characters through Greedy Attack.

This paper addresses the problem of generating adversarial examples for discrete domains like text. They propose two simple techniques:
1) Greedy: two stage process- first stage involves finding the k words in the sentence/paragraph to perturb and second step changes the word in the positions identified in step 1.
2) Gumbel: first approach amortized over datasets where first and second steps are parametrized and learned over the dataset with the loss being the probability of flipping the decision.
Specifically, for the Gumbel approach, the authors use the non-differentiable top-k-argmax output to train the module in the second step which is not ideal and it would be better to train both first and second steps jointly in an end-to-end differentiable manner.
The results show that Greedy approach is able to significantly affect the accuracy of the systems compared to other adversarial baselines. Mturk evaluation shows that for tasks like sentiment analysis, humans weren't as confused as the systems were when the selected words were changed which is encouraging. However, the Gumbel method performs poorly compared to other baselines.
Moreover, a thorough analysis of why Greedy is doing better than some gradient based adversarial attacks is needed in the paper because it is unclear what is causing their greedy approach to perform well; is it the two-stage nature of the process?
My major gripe with the paper is that it is egregiously difficult to read in parts and is poorly written. There are dangling conditional bars in many equations (5, 7, Greedy attack etc.), unclear "expectation (E)" signs and many other confusing notational choices which make the math difficult to parse. I am not even sure if those equations are correctly conveying the idea they are meant to convey. I found  the algorithms to be more clearly written and realize that the text in the models and equations is unnecessarily complicated. The argument about approximation to the objective by considering the i positions independently is not convincing and their is nothing in the paper to show if the assumption is reasonable.


Deep learning has been attracting enormous attention from academia as well as industry due to its great success in many artificial intelligence applications. As more applications are developed, the need for implementing a complex neural network model on an energy-limited edge device becomes more critical. To this end, this paper proposes a new optimization method to reduce the computation efforts of convolutional neural networks. The method takes advantage of the fact that some convolutional operations are actually wasteful since their outputs are pruned by the following activation or pooling layers. Basically, a convolutional filter conducts a series of multiply-accumulate (MAC) operations. We propose to set a checkpoint in the MAC process to determine whether a filter could terminate early based on the intermediate result. Furthermore, a fine-tuning process is conducted to recover the accuracy drop due to the applied checkpoints. The experimental results show that the proposed method can save approximately 50% MAC operations with less than 1% accuracy drop for CIFAR-10 example model and Network in Network on the CIFAR-10 and CIFAR-100 datasets. Additionally, compared with the state-ofthe-art method, the proposed method is more effective on the CIFAR-10 dataset and is competitive on the CIFAR-100 dataset.

This paper motivates itself by observing that not all convolutional weights are required to make an accurate prediction. In the introduction the authors envision a system similar to a cascaded classifier [Viola and Jones 2001] (I draw this conclusion not the paper). However the wording of the introduction is not clear or it does not align with what is presented in the paper.
The approach in the paper does not perform early stopping dynamically during the feedforward phase. The approach removes weights which do not impact the accuracy after training has completed and the fine tunes the resulting network.
The clarity of the introduction must be addressed however the work is still interesting. I recommend the authors try to make the introduction as accessible as possible.
Also there are very general statements like "The activation layer introduces non-linearity into the system for obtaining better accuracy." which do not contribute to the message of the paper. The paper will be fine without these statements. Shorter is better.
Section 3 is good as a motivating example. However the conclusion “Thus, our focus is to develop an effective method of choosing a good intermediate result for saving more MAC operations with less accuracy drop” is not very clear. More insights written hear would be better.
One major flaw is that no analysis with respect to time of computation was performed. GPUs offer the advantage of being optimized for convolutions so it is possible that there is no speedup. Because of this it is unclear if the method would save time. The results clearly show individual computations (MACs) are reduced but it is not clear how this correlates with wall clock time.
Why do you start with the centre layers? I understand the heuristic you’re using, that the middle layers won’t have high or low-level features, and that they won’t break the other layers as badly if you modify them, but I feel like this is core to your method and it’s not adequately justified. I’d like to see some experiments on that and whether it actually matters to the outcome. Also, you don’t say if you start at the middle then go up a layer, or down a layer. I think this matters since your proposed C10-Net has only 3 convolutional layers.
All the filters in the same layer share a common checkpoint. Is that a good idea? What is the cost of doing this on a per-filter level? What is the cost on a per-layer level? Discussing runtime estimates for the added computation at training would make it more clear what the cost of the method is.
In section 4.4.3. you mention that the majority of weight distributions in CNNs follow the Gaussian manner. Could you cite something of this? You might also want to move that in Step 1 (section 4.1.), since it seems to be your motivation for selection of checkpoint locations (5% and 32%) and I had no idea why you selected those values at that point.
Typos:
Typo on page 3: “exploits redundancies inter feature maps to prune filters and feature maps”
Structural:
Maybe section 4.3 should be part of the description of Section 4. Proposed Method, not its own subsection.
Maybe table 2 should go at the end of section 4.4.1, because you describe it the error and columns in section 4.4.1.


Value Propagation Networks
We present Value Propagation (VProp), a set of parameter-efficient differentiable planning modules built on Value Iteration which can successfully be trained using reinforcement learning to solve unseen tasks, has the capability to generalize to larger map sizes, and can learn to navigate in dynamic environments. We show that the modules enable learning to plan when the environment also includes stochastic elements, providing a cost-efficient learning system to build low-level size-invariant planners for a variety of interactive navigation problems. We evaluate on static and dynamic configurations of MazeBase grid-worlds, with randomly generated environments of several different sizes, and on a StarCraft navigation scenario, with more complex dynamics, and pixels as input.

Update:
I thank the authors for the response. Unfortunately, the response does not mention modifications made to the paper according to the comments. According to pdfdiff, modifications to the paper are very minor, and none of my comments are addressed in the paper. I think the paper shows good results, but it could very much benefit from improved presentation and evaluation. I do recommend acceptance, but if the authors put more work in improving the paper, it could have a larger impact.
------
The paper proposes a learnable planning model based on value iteration. The proposed methods can be seen as modifications of Value Iteration Networks (VIN), with some improvements aimed at improving sample efficiency and generalization to large environment sizes. The method is validated on gridworld-type environments, as well as on a more complex StarCraft-based domain with raw pixel input.
Pros:
1) The topic of the paper is interesting: combining the advantages of learning and planning seems like a promising direction to achieving adaptive and generalizable systems.
2) The presentation is quite good, although some details are missing.
3) The proposed method can be effectively trained with reinforcement learning and generalizes well to much larger environments than trained on. It beats vanilla VIN by a large margin. The MVProp variant of the method is especially successful.
Cons:
1) I would like to see a more complete discussion of the MVProp method. Propagation of only positive rewards seems like somewhat of a hack. Is this a general solution or is it only applicable to gridworld navigation-type tasks? Why? If not, is the area of applicability of MVProp different from VProp? Also, is the area of applicability of VProp different from VIN? It’s important to discuss this in detail.
2) I wonder how would the method behave in more realistic gridworld environments, for instance similar in layout to those used in RL navigation literature (DMLab, ViZDoom, MINOS, etc). The presented environments are quite artificial and seem to basically only require “obstacle avoidance”, not so much deliberate long-distance planning.
3) Some details are missing. For instance, I was not able to find the exact network architectures used in different tasks.
Related to this, I was confused by the phrase “As these new environments are not static, the agent needs to re-plan at every step, forcing us to train on 8x8 maps to reduce the time spent rolling-out the recurrent modules.” I might be misunderstanding something, but is there any recurrent network in VProp? Isn’t it just predicting the parameters once and then rolling our value iteration forward without any learning? Is this so time-consuming?
4) Why does the performance even of the best method not reach 100% even in the simpler environments in Figure 2? Why is the performance plateauing far from 100% in the more difficult case? It would be interesting to see more analysis of how the method works, when it fails, and which parts still need improvement. On a related topic, it would be good to see more qualitative results both in MazeBaze and StarCraft - in the form of images or videos.
5) Novelty is somewhat limited: the method is conceptually similar to VIN.
To conclude, I think the paper is interesting and the proposed method seems to perform well in the tested environments. I am quite positive about the paper, and I will gladly raise the rating if my questions are addressed satisfactorily.


NOODL: PROVABLE ONLINE DICTIONARY LEARNING AND SPARSE CODING
We consider the dictionary learning problem, where the aim is to model the given data as a linear combination of a few columns of a matrix known as a dictionary, where the sparse weights forming the linear combination are known as coefficients. Since the dictionary and coefficients, parameterizing the linear model are unknown, the corresponding optimization is inherently non-convex. This was a major challenge until recently, when provable algorithms for dictionary learning were proposed. Yet, these provide guarantees only on the recovery of the dictionary, without explicit recovery guarantees on the coefficients. Moreover, any estimation error in the dictionary adversely impacts the ability to successfully localize and estimate the coefficients. This potentially limits the utility of existing provable dictionary learning methods in applications where coefficient recovery is of interest. To this end, we develop NOODL: a simple Neurally plausible alternating Optimization-based Online Dictionary Learning algorithm, which recovers both the dictionary and coefficients exactly at a geometric rate, when initialized appropriately. Our algorithm, NOODL, is also scalable and amenable for large scale distributed implementations in neural architectures, by which we mean that it only involves simple linear and non-linear operations. Finally, we corroborate these theoretical results via experimental evaluation of the proposed algorithm with the current state-of-the-art techniques.

The paper considers the problem of dictionary learning. Here the model that we are given samples y, where we know that y = Ax where A is a dictionary matrix, and x is a random sparse vector. The goal is typically to recover the dictionary A, from which one can also recover the x under suitable conditions on A. The paper shows that there is an alternating optimization-based algorithm for this problem that under standard assumptions provably converges exactly to the true dictionary and the true coefficients x (up to some negligible bias).
The main comparison with prior work is with [1]. Both give algorithms of this type for the same problem, with similar assumptions (although there is some difference; see below). In [1], the authors give two algorithms: one with a better sample complexity than the algorithm presented here, but which has some systematic, somewhat large, error floor which it cannot exceed, and another which can obtain similar rates of convergence to the exact solution, but which requires polynomial sample complexity (the explicit bound is not stated in the paper). The algorithm here seems to build off of the former algorithm; essentially replacing a single hard thresholding step with an IHT-like step. This update rule is able to remove the error floor and achieve exact recovery. However, this makes the analysis substantially more difficult.
I am not an expert in this area, but this seems like a nice and non-trivial result. The proofs are quite dense and I was unable to verify them carefully.
Comments:
- The analysis in [1] handles the case of noisy updates, whereas the analysis given here only works for exact updates. The authors claim that some amount of noise can be tolerated, but do not quantify how much.
- A.4 makes it sound like eps_t needs to be assumed to be bounded, when all that is required is the bound on eps_0.
[1] Arora, S. Ge, R., Ma, T. and Moitra, A. Simple, Efficient, and Neural Algorithms for Sparse Coding. COLT 2015.


LEARNING WHAT YOU CAN DO BEFORE DOING ANYTHING
Intelligent agents can learn to represent the action spaces of other agents simply by observing them act. Such representations help agents quickly learn to predict the effects of their own actions on the environment and to plan complex action sequences. In this work, we address the problem of learning an agent's action space purely from visual observation. We use stochastic video prediction to learn a latent variable that captures the scene's dynamics while being minimally sensitive to the scene's static content. We introduce a loss term that encourages the network to capture the composability of visual sequences and show that it leads to representations that disentangle the structure of actions. We call the full model with composable action representations Composable Learned Action Space Predictor (CLASP). We show the applicability of our method to synthetic settings and its potential to capture action spaces in complex, realistic visual settings. When used in a semi-supervised setting, our learned representations perform comparably to existing fully supervised methods on tasks such as action-conditioned video prediction and planning in the learned action space, while requiring orders of magnitude fewer action labels.

The paper proposes a Variational IB based approach to learn action representations directly from video of actions being taken. The basic goal of the work is to disentangle the dynamic parts of the scene in the video from the static parts and only capture those dynamic parts in the representation. Further, a key property of these learned representations is that they contain compositional structure of actions so as to their cumulative effects. The outcome of such a method is better efficiency of the subsequent learning methods while requiring lesser amount of action label videos.
To achieve this, the authors start with a previously proposed video prediction model  that uses variational information bottleneck to learn minimal action representation. Next, this model is augmented with composability module where in latent samples across frames are composed into a single trajectory and is repeated in a iterative fashion and again the minimal representation for the composed action space is learned using IB based objective. The two objectives are learned in a joint fashion. Finally, they use a simple MLP based bijection to learn the correspondence between actions and their latent representations. Experiments are done on two datasets - reacher and BAIR - and evaluation is reported for action  conditioned video prediction and visual servoing.
- The paper is well written and provides adequate details to understand the flow of the material.
- The idea of learning disentangled representation is being adopted in many domains and hence this contribution is timely and very interesting to the community.
- The overall motivation of the paper to emulate how humans learn by looking at other's action is very well taken. Being able to learn from only videos is a nice property especially when the actual real world environment is not accessible.
- High Performance in terms of error and number of required action labeled videos demonstrates the effectiveness of the approach.
However, there are some concerns with the overall novelty and some technical details in the paper:
- It seems the key contribution of the paper is to add the L_comp part to the already available L_pred part in Denton and Fergus 2018. The trick use to compose the latent variables is not novel and considering that variational IB is also available, the paper lacks overall novelty. A better justification and exposition of novelty in this paper is required.
- Two simple MLP layers for bijection seems very adhoc. I am not able to see why such a simple bijection would be able to map the disentangled composed action representations to the actual actions. It seems it is working from the experiments but a better  analysis is required on how such a bijection is learned and if there are any specific properties of such bijection such that it will work only in some setting. Will the use of better network improve the learned bijection?
- While videos are available, Figures in the paper itself are highly unreadable. I understand the small figures in main paper but it should not be an issue to use full pages for the figure on appendix.
- Finally, it looks like one can learn the composed actions (Right + UP) representation while being not sensitive to static environment. If that is the case, does it work on the environment where except the dynamic part everything else is completely different? For example, it would be interesting to see if a model is trained where the only change in environment is a robot's hand moving in 4 direction while everything else remaining same. Now would this work, if the background scene is completely changed while keeping the same robot arm?


Deep neural networks have been recently demonstrated to be vulnerable to backdoor attacks. Specifically, by altering a small set of training examples, an adversary is able to install a backdoor that can be used during inference to fully control the model's behavior. While the attack is very powerful, it crucially relies on the adversary being able to introduce arbitrary, often clearly mislabeled, inputs to the training set and can thus be detected even by fairly rudimentary data filtering. In this paper, we introduce a new approach to executing backdoor attacks, utilizing adversarial examples and GAN-generated data. The key feature is that the resulting poisoned inputs appear to be consistent with their label and thus seem benign even upon human inspection.

This paper investigates an interesting problem, backdoor attack against neural networks. The main idea is to add a watermark pattern to the corners of the training images, so that the classifier is guided to leverage the watermark as a discriminative cue as opposed to the real content of the image. At the test stage, one can hence manipulate the classifier’s predictions by adding the watermark to the test images.
This paper is heavily built upon Gu et al. (2017)’s work. It shows that Gu et al. (2017)’s method can be easily defended by a data sanitization algorithm. To improve Gu et al.’s work, the authors propose to add watermark patterns to the adversarial examples or examples interpolated in GAN’s latent space. The intuition is that these examples are adversarial and hard to learn, forcing the classifier to focus on the watermark pattern instead.
It is an interesting idea and an intuitive improvement over (Gu et al. 2017). However, the implementation of the idea could be improved. This paper does not propose any new attack algorithms. Instead, it investigates an existing adversarial attack method and the GAN based interpolation for the purpose of backdoor attack. As experiments are conducted on small-scale datasets, it is unclear how effective the improved backdoor attack is. Moreover, one of the main disadvantages of the proposed attack method is that simple data augmentation techniques, especially random cropping, can successfully defend against the attack.
The quality of the paper writing could be improved. I had to read the paper more than twice and check the references now and then in order to understand some claims of the paper. The paper’s lack of clarity was actually also raised by probably one of the coauthors of the paper; see the comment “Dimitris: clarify this point” on Page 11. Please find some concrete suggestions below.
- Figure 1 is visually not appealing at all. Perhaps find better illustrative examples.
- It is worth considering to add a separate section/paragraph to describe the details of Gu et al. (2017)’s method, given that this paper is heavily built upon Gu et al. (2017)’s work.
- It was unclear what the “reduced amplitude backdoor trigger” means until Section 4. If a context-dependent term has to be used in the introduction, explain it or refer the readers to the right place of the paper.
- Merge Sections 4.3—4.5 with the experiment section (Section 5). The results of Section 4.3–4.5 are out of context without any explanation about the experiment setups.
I have some concerns about Section 3, which is the main motivation of this work. As the authors noted in Appendix A that Gu et al.’s method works well with as few as 75 poised examples, the proposed sanitization algorithm would not be able to fail Gu et al.’s method by only identifying 20 out of 100 poised examples.
How to control the parameter $\tau$ so that the perturbation appears plausible to humans?


Recently, deep neural networks have achieved superhuman performance in various games such as Go, chess and Shogi. Compared to Go, China Competitive Poker, also known as Dou dizhu, is a type of imperfect information game, including hidden information, randomness, multi-agent cooperation and competition. It has become widespread and is now a national game in China. We introduce an approach to play China Competitive Poker using Convolutional Neural Network (CNN) to predict actions. This network is trained by supervised learning from human game records. Without any search, the network already beats the best AI program by a large margin, and also beats the best human amateur players in duplicate mode.

This paper provides a system to play CCP using some deep learning. The system consists of  three modules - the bid module, which is rule based, and the policy and kicker networks, which are simple convolutional neural networks. The authors use a dataset of 8 million game records consisting of 80 million state action pairs, and train the network in a supervised fashion. The resulting model is able beat MicroWe, the current state of the art in playing CCP, and even are able to beat a few "top amateur players"
- Why is the bid module also not learned? It seems like the feature set for the bid module is fairly simple, and a linear or MLP can do fairly well compared to a rule based module.
- It's not clear that separating the policy and kicker networks would be more advantageous than combining them. Thousands of actions is not a too large number - language modeling work routinely deals with outputting many more classes than that.
- Were the convolutions chosen 1D, 2D, or 3D? The figure seems to imply that the convolutions were over the XZ dimensions, with Y as the channel dimension. If so, this doesn't make too much sense to be, since the Z dimension is not uniform - the last index is all unseen cards, which is significantly more than the middle indices of "what was played in this round". There shouldn't be a lot of translational invariance in the Z dimension. I'm also not convinced that translational invariance is helpful in the X dimension.
- There were no comparisons with baseline models or different model architectures. I would like to see some results on the same structure, but with an Linear model, MLP or LSTM across the time dimension, or search through different types of convolutional networks.
- What hyperparameters were searched through in the learning process?
- Missing citations for MicroWe being the best CCP AI, and citations for the accomplishments of the top amateur players.
- How far away are the top amateur players from professional players? Please provide some context on how far this system is from solving CCP.
- Fig 3, 4 should just say #of games instead of "iteration"
This paper shows that one choice for a supervised learning system on a CCP game database can achieve amateur level human play. It does not give insight to why the system was designed this way, why the model choices were made, and how good simpler baselines might be able to achieve. The paper is not clearly written enough, and does not provide enough scientific value to be accepted to the conference.