OPTIMAL COMPLETION DISTILLATION FOR SEQUENCE LEARNING
We present Optimal Completion Distillation (OCD), a training procedure for optimizing sequence to sequence models based on edit distance. OCD is efficient, has no hyper-parameters of its own, and does not require pretraining or joint optimization with conditional log-likelihood. Given a partial sequence generated by the model, we first identify the set of optimal suffixes that minimize the total edit distance, using an efficient dynamic programming algorithm. Then, for each position of the generated sequence, we define a target distribution that puts an equal probability on the first token of each optimal suffix. OCD achieves the state-of-theart performance on end-to-end speech recognition, on both Wall Street Journal and Librispeech datasets, achieving 9.3% and 4.5% word error rates, respectively.

The authors propose an alternative approach to training seq2seq models, which addresses concerns about exposure bias and about the typical MLE objective being different from the final evaluation metric. In particular, the authors propose to use a dynamic program to compute the optimal continuations of predicted prefixes (during training) in terms of edit distance to the true output, and then use a per-token cross entropy loss, with a target distribution that is uniform over all optimal next-tokens. The authors conduct a number of experiments, and show that this scheme allows them to attain state-of-the-art performance on end-to-end speech recognition, and that they can moreover do this without needing to pretrain the model with the MLE objective.
This is a very nice paper; it is generally well written, it gets excellent results, and it contains a comprehensive set of experiments and analysis. It is also quite exciting to see an approach to alleviating exposure bias that does not require pretraining with MLE. Accordingly, my suggestions mainly relate to the presentation and related work:
 - It seems a bit strange to argue that the proposed approach doesn't increase the time complexity over MLE. While technically true (as the authors note) if the vocabulary is bigger than the sequence lengths, the difference in (on policy) training time will presumably be felt when dealing with very long sequences, or with cases where the number of labels per time-step is small, like in character-level generation or in seq2seq style sequence labeling.
 - I think it's difficult to argue that the proposed approach isn't essentially a modification of imitation learning/learning-to-search algorithms like, say, AggreVaTe or LOLS (Chang et al., ICML 2015). As far as I can tell, the only differences are that cross entropy is used rather than a cost-sensitive classifier, and, perhaps, that the training is done in minibatches (with no aggregation).

 - Relatedly, while it is interesting that the loss uses all the optimal completion tokens, it should be noted that there is much work in transition-based parsing that adopts a learning-to-search approach and uses losses that incorporate multiple optimal next-predictions as given by a "dynamic oracle"; see Goldberg and Nivre (COLING, 2012) and others.
 - I think it's also worth noting that training approaches like MIXER and others can target arbitrary rewards (and not just those where we can efficiently compute optimal next-steps), and so the proposed approach is a compelling competitor to MIXER-like approaches on problems such as machine translation or image captioning only to the extent that training with edit-distance is useful for such problems. Do you have a sense of whether training with edit-distance does indeed improve performance on such tasks?
Pros:
- well written and interesting
- good experiments, results, and analysis
Cons:
 - perhaps slightly more similar to previous work than is argued
Update after author response: thanks for your response; I think the revised paper largely addresses my comments and those of the other reviewers, and I continue to hope it is accepted. Here are two small notes on the related work section of the revised paper:
- In distinguishing OCD from DAgger, you note that the optimal policy is computed rather than provided at training time. In fact, structured prediction applications of SEARN (Daume III et al., 2009, which should also be cited) and DAgger often have this flavor too, such as when using them for sequence labeling (where optimal continuations are calculated based on Hamming distance).
- Please include a reference to Goldberg and Nivre's (2012) dynamic oracle work.


Understanding the groundbreaking performance of Deep Neural Networks (DNNs) is one of the greatest challenges to the scientific community today. In this work we introduce an information theoretic viewpoint on the behavior of deep networks optimization processes and their generalization abilities. Specifically, we study DNNs on the Information Plane, the plane of the mutual information between each layer with and the input variable, and with the desired label, during the training dynamics. We show that the training of the network is characterized by a rapid increase in the mutual information (MI) between the layers and the target label, followed by a longer decrease in the MI between the layers and the input variable.vFurther, we explicitly show that these two fundamental information-theoretic quantities govern the generalization error of the network, by introducing a new generalization-gap bound that is exponential in the input representation compression. The analysis focuses on typical patterns of large-scale problems. For this purpose, we introduce a novel analytic bound on the mutual information between consecutive layers in the network. An important consequence of our analysis is a super-linear boost in training time with the number of nondegenerate hidden layers, demonstrating the computational benefit of the hidden layers.

This paper interprets the optimization of deep neural networks in terms of a two phase process: first a drift phase where gradients self average, and second a diffusion phase where the variance is larger than the square of the mean. As argued by first by Tishby and Zaslavsky and then by Shwartz-Ziv and Tishby (arxiv:1703.00810), the first phase corresponds to the hidden layers becoming more informative about the labels, and the second phase corresponds to a compression of the hidden representation keeping the informative content relatively fixed as in the information bottleneck of Tishby, Pereira, and Bialek.
A lot of this paper rehashes discussion from the prior work and does not seem sufficiently original. The main contribution seems to be a bound that is supposed to demonstrate representation compression in the diffusion phase. The authors further argue that this shows that adding hidden layers lead to a boosting of convergence time.
Furthermore, the analytic bound relies on a number of assumptions that make it difficult to evaluate. One example is using the continuum limit for SGD (1), which is very popular but not necessarily appropriate. (See, e.g., the discussion in section 2.3.3 in arxiv:1810.00004.)
Additionally, there has been extensive discussion in the literature regarding whether the results of Shwartz-Ziv and Tishby (arxiv:1703.00810) hold in general, centering in particular on whether there is a dependence on the choice of the hyperbolic tangent activation function. I find it highly problematic that the authors continue to do all their experiments using the hyperbolic tangent, even though they claim their analytic bounds are supposed to hold for any choice of activation. If the bound is general, why not include experimental results showing that claim? The lack of discussion of this point and the omission of such experiments is highly suspicious.
Perhaps more importantly, the authors do not even mention or address this contention or even cite this Saxe et al. paper (https://openreview.net/forum?id=ry_WPG-A-) that brings up this point. They also cite Gabrie et al. (arxiv:1805:09785) as promising work about computing mutual information for deep networks, while my interpretation of that work was pointing out that such methods are highly dependent on choices of binning or regulating continuous variables when computing mutual informations. In fact, I don't see any discussion at all this discretization problem, when it seems absolutely central to understanding whether there is a sensible interpretation of these results or not.
For all these reasons, I don't see how this paper can be published in its present form.


Modularity is a key feature of deep learning libraries but has not been fully exploited for probabilistic programming. We propose to improve modularity of probabilistic programming language by offering not only plain probabilistic distributions but also sophisticated probabilistic model such as Bayesian nonparametric models as fundamental building blocks. We demonstrate this idea by presenting a modular probabilistic programming language MXFusion, which includes a new type of re-usable building blocks, called probabilistic modules. A probabilistic module consists of a set of random variables with associated probabilistic distributions and dedicated inference methods. Under the framework of variational inference, the pre-specified inference methods of individual probabilistic modules can be transparently used for inference of the whole probabilistic model. We demonstrate the power and convenience of probabilistic modules in MXFusion with various examples of Gaussian process models, which are evaluated with experiments on real data.

In this paper authors present a new Probabilistic Programming Language (PPL) MXFusion. Similarly to the languages for the deep learning (TensorFlow, PyTorch, etc.), this language introduce probabilistic modules that are used as building blocks for complex probabilistic models. Introducing modularity to the probabilistic programming, raises the problem of inference for probabilistic models. Since, we cannot obtain the exact solution on practice we have to resort to approximate inference methods. The approximate inference methods can be either generic, thus, being suitable for many probabilistic models but resulting in poor approximation, or specific, thus, having good approximation quality, but only for specific probabilistic models. Authors propose to address this problem by encapsulating specific inference methods in corresponding probabilistic modules. Doing so, one can perform approximate inference for every module with the best suitable inference technique. Authors demonstrate interface of MXFusion for three well known probabilistic models: Bayesian linear regression, deep kernel learning, Bayesian Gaussian process latent variable model.
Approaching the problem of building complex probabilistic models by introducing modular PPL is an important direction of study. But, regarding this paper I have the following concerns.
- In my opinion, the structure of the paper can be greatly improved. From general words about modularity and approximate inference authors dive to the very specific cases of probabilistic models. Following such structure, authors don’t give a clear answer to the following questions. Why the paradigm of encapsulating inference methods in probabilistic modules is legitimate for constructing complex probabilistic models? What inference methods and probabilistic models can we use as building blocks? Do we need to be aware of specific inference methods that are encapsulated or we can use any blocks in any order as we do in deep learning frameworks?
- Novelty of that paper is the new design of PPL. That is an interesting and important question for the community, but maybe ICLR paper is not the best format to present such kind of novelty.
- From the specific examples in the paper, legitimacy of such modular structure is clear only for variational inference (that seems to be a common knowledge) and variational approximation of gaussian processes. But the application area of MXFusion remains unclear. Verbatim examples of code for the specific examples doesn’t make the difference between MXFusion and other PPLs clear, because it can be treated as encapsulation of the code into some classes, that can be implemented in other languages as well.
- Comparison with other frameworks can be improved. In experimental section authors provide comparison with GPy framework in terms of RMSE and log-likelihood for gaussian process with 50 inducing points. As I understood both frameworks use the same inference methods and achieve the same performance, so the experiment can be considered as sanity check for MXFusion. The paper could benefit from comparison between different inference methods and providing benchmarks for inference time.
Overall, the paper proposes a new PPL that is an important direction of study, but have several drawbacks and conference paper format is not the best way to present such kind of novelty.
Typos:
- Page 1, “despite the different of DNNs…” -> “despite the difference of DNNs…”?
- Page 2, missing reference of the section
- Page 2, section 3, “... sightly different form.” -> “... slightly different form”?


AN EMPIRICAL STUDY OF EXAMPLE FORGETTING DURING DEEP NEURAL NETWORK LEARNING
Inspired by the phenomenon of catastrophic forgetting, we investigate the learning dynamics of neural networks as they train on single classification tasks. Our goal is to understand whether a related phenomenon occurs when data does not undergo a clear distributional shift. We define a "forgetting event" to have occurred when an individual training example transitions from being classified correctly to incorrectly over the course of learning. Across several benchmark data sets, we find that: (i) certain examples are forgotten with high frequency, and some not at all; (ii) a data set's (un)forgettable examples generalize across neural architectures; and (iii) based on forgetting dynamics, a significant fraction of examples can be omitted from the training data set while still maintaining state-of-the-art generalization performance.

UPDATE 2 (Nov 19, 2018): The paper has improved very substantially since the initial submission, and the authors have addressed almost all of my comments. I have therefore increased my score to an 8 and recommend acceptance.
------------------------------------------------------------------------------------------------------------------------------
UPDATE (Nov 16, 2018) : In light of the author response, I have increased my score to a 6.
------------------------------------------------------------------------------------------------------------------------------
This paper aims to analyze the extent to which networks learn to correctly classify specific examples and then “forget” these examples over the course of training. The authors provide several examples of forgettable and unforgettable examples, demonstrating, among other things, that examples with noisy examples are more forgettable and that a reasonable fraction of unforgettable examples can be removed from the training set without harming performance.
The paper is clearly written, and the work is novel -- to my knowledge, this is the first investigation of example forgetting over training. There are an interesting and likely important set of ideas here, and portions of the paper are quite strong -- in particular, the experiment demonstrating that examples with noisy examples are more forgettable is quite nice. However, there are several experimental oversights which make this paper difficult to recommend for publication in its current form.
Major points:
1) The most critical issue is with the measurement of forgetting itself: the authors do not take into account the chance forgetting rate in any of their experiments. Simply due to chance, some examples will be correctly labeled at some point in training (especially in the datasets analyzed, which only contain 10 classes). This makes it difficult to distinguish whether a “forgotten” example was actually ever learned in the first place. In order to properly ground this metric, measurements of chance forgetting rates will be necessary (for example, what are the forgetting rates when random steps are taken at each update step?).
2) Were the networks trained on MNIST, permutedMNIST, and CIFAR-10 trained for the same number of epochs? Related to point 1, the forgetting rate should increase with the number of epochs used in training as the probability of each example being correctly classified should increase. If the CIFAR-10 models were trained for more epochs, this would explain the observation that more CIFAR-10 examples were “forgettable.”
3) In the experiment presented in Figure 4b, it is difficult to tell whether the never forgotten set suffers less degradation in the third training regime because the examples were never forgotten or because the model had twice has much prior experience. Please include a control where the order is flipped (e.g., forgotten, never forgotten, forgotten in addition to the included never forgotten, forgotten, never forgotten order currently present).
4) The visual inspection of forgettable and unforgettable examples in Figure 2 is extremely anecdotal, and moreover, do not even appear to clearly support the claims made in the paper.
Minor points:
1) In the discussion of previous studies which attempted to assess the importance of particular examples to classification decisions, a citation to [1] should be added.
2) The point regarding similarity across seeds is absolutely critical (especially wrt major comment 1) , and should be included earlier in the paper and more prominently.
3) The histograms in Figure 1 are misleading in the cropped state. While I appreciate that the authors included the full histogram in the supplement, these full histograms should be included in the main figure as well, perhaps as an inset.
4) The inclusion of a space after the commas in numbers (e.g., 50, 245) is quite confusing, especially when multiple numbers are listed as in the first line on page 4.
[1] Koh, Pang Wei and Percy Liang. “Understanding Black-box Predictions via Influence Functions.” ICML (2017).


We study the joint distribution matching problem which aims at learning bidirectional mappings to match the joint distribution of two domains. This problem occurs in unsupervised image-to-image translation and video-to-video synthesis tasks, which, however, has two critical challenges: (i) it is difficult to exploit sufficient information from the joint distribution; (ii) how to theoretically and experimentally evaluate the generalization performance remains an open question. To address the above challenges, we propose a new optimization problem and design a novel Joint Wasserstein Auto-Encoders (JWAE) to minimize the Wasserstein distance of the joint distributions in two domains. We theoretically prove that the generalization ability of the proposed method can be guaranteed by minimizing the Wasserstein distance of joint distributions. To verify the generalization ability, we apply our method to unsupervised video-to-video synthesis by performing video frame interpolation and producing visually smooth videos in two domains, simultaneously. Both qualitative and quantitative comparisons demonstrate the superiority of our method over several state-of-the-arts.

This paper studies the joint distribution matching problem where given data samples in two different domains, one is interested in learning a bi-directional mapping between unpaired data elements in these domains. The paper proposes a joint Wasserstein auto-encoder (JWAE) to solve this problem. The paper shows that under the decomposable cost metric and deterministic decoding maps, the optimization problem associated with the JWAE formulation can be reduced to a tractable optimization problem. The paper also establishes a generalization bound for the JWAE formulation. Finally, the paper conducts an experimental evaluation of the proposed solution with the help of a video-to-video synthesis problem and show improved performance as compared to the existing results in the literature.
Overall, the reviewer finds that the paper considers an important problem and proposes some interesting ideas to tackle the problem. However, in its current form, there is a large scope for improvement in the presentation of the paper. The paper is full of errors/typos which make it an extremely difficult read (see my comments below). That said the paper fairs quite well as compared to other existing methods. Since the reviewer is not very much familiar with this field, the reviewer leaves it to the other reviewers to decide the significance of these results.
Pros:
1) The paper aims to provide a theoretical treatment of the joint distribution matching problem which has many interesting applications, including image-to-image translation and video-to-video synthesis.
2) The proposed method in the paper had good empirical performance on the real world datasets.
Cons:
The paper is very poorly written with many typos and (possibly) mistakes. Some of the comments in this direction are as follows.
1) The paper does not formally define the underlying problem before diving into the details of the proposed solution. The authors only informally talk about the problem in the introduction. Given that the ICLR has a wide audience, it would have been nice if the authors have made the presentation of the paper self-contained.
2) In the same vein, the paper talks about many important quantities without introducing them first. E.g., what are $E_{A}(f^*)$, $E_B^g(f^*)$ etc. in the statement of Theorem 2? These quantities are first defined inside the proofs in the supplementary material!
3) Some of the notation in the paper is also very confusing. For example, cross-domain mapping have two different sets of notations. $(E1oG2, E2oG)$ in Sec. 4.2 and $(G2oE1, G1oE2)$ in Section 5. It should be latter. Similarly, Sec. 4.2 refers to $E1oG1$ and $E2oG2$ as auto-encoders, which should be $G1oE1$ and $G2oE2$, respectively. In Sec. 3, the authors refer to $N$ and $M$ as the number of samples in the $X$ and $Y$ domain, respectively. This is then reversed in Theorem 2 and 4. These are only a small list of large number of such typos. Also, what is the notion defined in the last line of Sec. 3?
4) It is not clear to me why the sets $Q_1$ and $Q_2$ in Theorem 1 are define in their current forms. In particular, it is not clear why the equality hold in Eq. (17) in the proof of Theorem 1.
5) One line in the proof of Lemma 1 says, "Specifically, we choose its equality, then we have". Could the authors elaborate on this?
6) Eq. (24) should be inequality?
7) Given that the authors write a regularized problem in (4). Does that mean now sets $Q_1$ and $Q_2$ are different from how they are defined in the statement of Theorem 1?
#########################
Post rebuttal: The authors have addressed most of my concerns regarding the poor presentation of the earlier version. I have updated my score.


Learning latent representations of nodes in graphs is an important and ubiquitous task with widespread applications such as link prediction, node classification, and graph visualization. Previous methods on graph representation learning mainly focus on static graphs, however, many real-world graphs are dynamic and evolve over time. In this paper, we present Dynamic Self-Attention Network (DySAT), a novel neural architecture that operates on dynamic graphs and learns node representations that capture both structural properties and temporal evolutionary patterns. Specifically, DySAT computes node representations by jointly employing self-attention layers along two dimensions: structural neighborhood and temporal dynamics. We conduct link prediction experiments on two classes of graphs: communication networks and bipartite rating networks. Our experimental results show that DySAT has a significant performance gain over several different stateof-the-art graph embedding baselines.

This is a well-written paper studying the important problem of dynamic network embedding. Please find below some pros and cons of this paper.
Pros:
* Studies the important problem of network embedding under a more realistic setting (i.e., nodes & edges evolve over time).
* Introduces an interesting architecture that uses two forms of attention: structural and temporal.
* Demonstrated the effectiveness of the temporal layers through additional experiments (in appendix) and also introduced a variant of their proposed approach which can be trained incrementally using only the last snapshot.
Cons:
* The authors compared against several dynamic & static graph embedding approaches. If we disregard the proposed approach (DySAT), the static methods seem to match and even, in some cases, beat the dynamic approaches on the compared temporal graph datasets. The authors should compare against stronger baselines for static node embedding, particularly GAT which introduced the structural attention that DySAT uses to show that the modeling of temporal dependencies is necessary/useful. Please see [1] for an easy way to train GCN/GAT for link prediction.
* There are actually quite a number of work done on network embedding on dynamic graphs including [2-4]. In particular, [2-3] support node attributes as well as the addition/deletion of nodes & edges. The author should also compare against these work.
* The concept of temporal attention is quite interesting. However, the authors do not provide more analysis on this. For one, I am interested to see how the temporal attention weights are distributed. Are they focused on the more recent snapshots? If so, can we simply retain the more relevant recent information and train a static network embedding approach? Or are the attention weights distributed differently?
[1] Modeling Polypharmacy Side Effects with Graph Convolutional Networks. Zitnik et. al. BioInformatics 2018.
[2] Attributed Network Embedding for Learning in a Dynamic Environment. Li et. al. In Proc. CIKM '17.
[3] Streaming Link Prediction on Dynamic Attributed Networks. Li et. al. In Proc. WSDM '18.
[4] Continuous-Time Dynamic Network Embeddings. Nguyen et. al. In Comp. Proc. WWW '18.


Deep Neutral Networks(DNNs) require huge GPU memory when training on modern image/video databases. Unfortunately, the GPU memory as a hardware resource is always finite, which limits the image resolution, batch size, and learning rate that could be used for better DNN performance. In this paper, we propose a novel training approach, called Re-forwarding, that substantially reduces memory usage in training. Our approach automatically finds a subset of vertices in a DNN computation graph, and stores tensors only at these vertices during the first forward. During backward, extra local forwards (called the Re-forwarding process) are conducted to compute the missing tensors between the subset of vertices. The total memory cost becomes the sum of (1) the memory cost at the subset of vertices and (2) the maximum memory cost among local re-forwards. Re-forwarding trades training time overheads for memory and does not compromise any performance in testing. We propose theories and algorithms that achieve the optimal memory solutions for DNNs with either linear or arbitrary computation graphs. Experiments show that Re-forwarding cuts down up-to 80% of training memory on popular DNNs such as Alexnet, VGG, ResNet, Densenet and Inception net.

I took a look at the revision.  I am glad to see that the authors clarified the meaning of "optimality" and added time complexity for each algorithm. The complexities of the algorithms do not seem great (3rd or 4th order polynomial of N) as they appear to be checking things exhaustively, but perhaps they are not a big issue since the decomposition algorithm is run only once and usually N is not huge. I wonder if more efficient algorithms exist.
It is also nice to see that the time overhead for one training step is not huge (last column of Table 1). While I still think it is better to see more complete training curves, the provided time overhead is a proxy.
I hope the authors can further improve the algorithm description, for example, the pseudo-code of Algorithm 1 is very verbal/ambiguous, and it would be better to have more implementation-friendly pseudo-code.
Despite the above-mentioned flaws, I think this work is still valuable in handling the memory consumption of arbitrary computation graph in a principled manner.
=============================================================================
This paper presents a method for reducing the memory cost of training DNNs. The main idea is to divide the computational graph into smaller components (close sets), such that the dependency between components is low, and so one can store only tensors at key vertices in each component during the forward pass. In the backward pass, one needs to re-forward the data within each close set for computing the gradient.
The idea is quite intuitive and the example in linear computational graph in Section 3 clearly demonstrates the basic idea. The main technical development is in the case of arbitrary computation graph (Section 4), where the authors explain how to divide the computational graph into different types of close sets. I have not read the proofs in appendix, but it is clear that the technical development is mainly in discrete math rather than machine learning. For this part, my suggestions are:
1. give better algorithm description: what are the inputs and outputs of each algorithm, for the examples in Figure 2-4 (and perhaps with costs for each vertex), which vertices do the final algorithm decide to store
2. analyze the complexity of each algorithm
3. provide clear definition of "optimality" and its proof: the authors mentioned in a few places about the method being "optimal", but the paper needs to be crystal clear about "for what problem is the method optimal", "optimal in what sense (the objective)".
The more concerning part is the experimental evaluation. While I believe that the proposed method can save memory cost, there is no result on the additional time cost for re-forwarding. Therefore, it is not clear if it is worth the extra complication of using re-forwarding in an end-to-end sense: the authors motivated in Section 1 that saving memory cost allows one to use larger mini-batches, and an important benefit of large mini-batch is to accelerate training, see, e.g.,
Goyal et al. Accurate, large minibatch SGD: Training imagenet in 1 hour.
Hoffer et al. Train longer, generalize better: Closing the generalization gap in large batch training of neural networks.
It is less desirable to use re-forwarding it if it causes significant latency in the backward pass and training. An good demonstration of the memory vs. time tradeoff would be the training curves of loss vs. running time.
The other detail I would appreciate is how the authors implemented the algorithm, and whether the proposed method is likely to be deployed on popular ML frameworks, which would improve the significance of the work.


HIERARCHICAL INTERPRETATIONS FOR NEURAL NET- WORK PREDICTIONS
Deep neural networks (DNNs) have achieved impressive predictive performance due to their ability to learn complex, non-linear relationships between variables. However, the inability to effectively visualize these relationships has led to DNNs being characterized as black boxes and consequently limited their applications. To ameliorate this problem, we introduce the use of hierarchical interpretations to explain DNN predictions through our proposed method: agglomerative contextual decomposition (ACD). Given a prediction from a trained DNN, ACD produces a hierarchical clustering of the input features, along with the contribution of each cluster to the final prediction. This hierarchy is optimized to identify clusters of features that the DNN learned are predictive. We introduce ACD using examples from Stanford Sentiment Treebank and ImageNet, in order to diagnose incorrect predictions, identify dataset bias, and extract polarizing phrases of varying lengths. Through human experiments, we demonstrate that ACD enables users both to identify the more accurate of two DNNs and to better trust a DNN's outputs. We also find that ACD's hierarchy is largely robust to adversarial perturbations, implying that it captures fundamental aspects of the input and ignores spurious noise.

This paper proposes a novel approach to explain neural network predictions by learning hierarchical representations of groups of input features and their contribution to the final prediction. The proposed method is a straightforward extension of the contextual decomposition work by (Murdoch et. al. 2018) which estimates feature interpretability for LSTMs. This work extends (Murdoch et. al. '18) to more general NN architectures and further employs agglomerative clustering to identify groups of features-- as opposed to individual features--that are predictive of the output.
Results are shown using a LSTM trained on the standard Stanford sentiment task and a VCG DNN trained on ImageNet which show the superior performance of the proposed approach. In addition, the paper also provides some survey results where "humans" were asked to pick more interpretable models.
The paper is nicely written and puts itself nicely in context of the previous work. Though, I have several concerns:
1). Biggest concern: Conditioning on the (Murdoch et. al. 18) paper, the methodological novelty of the proposed approach is minimal. Though, the experimental gains on the vision and NLP tasks are nice.
2). It was unclear to me how the agglomerative algorithm (Algorithm 1) was run. That is, was it run as part of the LSTM estimation for instance for the sentiment task OR was it run post-hoc after getting the model estimates from LSTM? If it was run post-hoc then I am unsure if we can assume that the "agglomeratively grouped CD scores of individual features" are the same as the "CD scores for the groups/interactions of features" in terms of their contribution to the final prediction.
3). Though, the paper mentions several times regarding generalizing (Murdoch et. al. 18) to architectures other than LSTMs but still the experimental results on the sentiment task uses an LSTM as the model. It would have been nice to show the comparative strength of the proposed approach on a different architecture even for the sentiment task. (I understand that the paper uses a different DNN architecture for the vision task).
4). The paper talks several times about diagnosing why a model went wrong e.g. the "negation" in the case of the LSTM model in Figure 2, but never discusses the bigger and more interesting problem. How can we build an improved LSTM model for the sentiment task which classifies that incorrect prediction correctly?


FEED-FORWARD PROPAGATION IN PROBABILISTIC NEURAL NETWORKS WITH CATEGORICAL AND MAX LAYERS
Probabilistic Neural Networks deal with various sources of stochasticity: input noise, dropout, stochastic neurons, parameter uncertainties modeled as random variables, etc. In this paper we revisit a feed-forward propagation approach that allows one to estimate for each neuron its mean and variance w.r.t. all mentioned sources of stochasticity. In contrast, standard NNs propagate only point estimates, discarding the uncertainty. Methods propagating also the variance have been proposed by several authors in different context. The view presented here attempts to clarify the assumptions and derivation behind such methods, relate them to classical NNs and broaden their scope of applicability. The main technical contributions are new approximations for the distributions of argmax and max-related transforms, which allow for fully analytic uncertainty propagation in networks with softmax and max-pooling layers as well as leaky ReLU activations. We evaluate the accuracy of the approximation and suggest a simple calibration. Applying the method to networks with dropout allows for faster training and gives improved test likelihoods without the need of sampling.

* Summary
The authors focus on the problem of uncertainty propagation DNN. The authors claim two main contributions: they revisit the assumptions of the feed forward method (proposed by several authors as an inference method for BNNs based on ADF/EP) and proposed a new approximation for argmax/max based functions that allows to propagated the first two moments analytically.
* Comments:
The authors claim two main contributions: an analysis for the feed forward method (sections 2 and 3) previously proposed by several authors as an inference method for BNN based on ADF/EP, and a new method to propagate the uncertainty through argmax/max based operations (section 4).
Regarding the first contribution, I was expecting some new insights about the method that I did not find. I would suggest to focus on the second contribution and refactor this section as a background section. I would make it shorter, focusing on the representation of probabilities as latent variables trough a function, which is the important bit to understand the real contribution of the paper described in section 4. I would also remove some examples that do not seem critical to understand the rest of the paper and just increase its length.
The second contribution is quite novel. The authors propose a new approximation of argmax/max operations. The firstly proposed an approximation for argmax operations, e.g. latent variable view of the softmax, that avoids resorting to the normal cdf function that has numerical stability issues. Secondly, they suggest an approximation for max based operations, e.g. leaky relu, that again, does not depend on the gaussian cdf.
In the experimental section, the authors test:
a)	The accuracy of the proposed method approximating the posterior of the neurons
b)	End-to-end training benefits
In a) they use MC to collect the ground truth statistics and compare the proposed method (AP2) with a classical NN (AP1). The analysis is nice but I miss a comparison with other state-of-the-art methods. In particular, the authors claim that the novelty of their method compared to other feed-forward methods is that they can propagate the uncertainty through argmax/max operations analytically. They do not compare with these other feed forwards methods to show the benefit of this.  This is shown in the end-to-end training experiments; however, I would like to see a direct comparison with the classical paper (Hern´andez-Lobato & Adams, 2015). Finally, one of the justifications of the approximations that they propose is to avoid the numerical issues of the standard cdf. Have the authors compared with this, e.g. eq 18a, 18b? Using a robust implementation of the normal cdf/pdf function and further truncating them to avoid negative variances?
typo: Shortly before eq. 12, Should not S_{n-1} be defined as the softmax operation?


APPROXIMATING CNNS WITH BAG-OF-LOCAL- FEATURES MODELS WORKS SURPRISINGLY WELL ON IMAGENET
Deep Neural Networks (DNNs) excel on many complex perceptual tasks but it has proven notoriously difficult to understand how they reach their decisions. We here introduce a high-performance DNN architecture on ImageNet whose decisions are considerably easier to explain. Our model, a simple variant of the ResNet-50 architecture called BagNet, classifies an image based on the occurrences of small local image features without taking into account their spatial ordering. This strategy is closely related to the bag-of-feature (BoF) models popular before the onset of deep learning and reaches a surprisingly high accuracy on ImageNet (87.6% top-5 for 33 × 33 px features and Alexnet performance for 17 × 17 px features). The constraint on local features makes it straight-forward to analyse how exactly each part of the image influences the classification. Furthermore, the BagNets behave similar to state-of-the art deep neural networks such as VGG-16, ResNet-152 or DenseNet-169 in terms of feature sensitivity, error distribution and interactions between image parts. This suggests that the improvements of DNNs over previous bag-of-feature classifiers in the last few years is mostly achieved by better fine-tuning rather than by qualitatively different decision strategies.

This is an experimental paper that investigates how spatial ordering of patches influences the classification performances of CNNs. To do so, the authors design CNNs close to ResNets that almost only consist in a simple cascade of 1x1 convolutions, obtaining relatively small receptive field. It is an interesting read, and I recommend it as a valuable contribution to ICLR, that might lead to nice future works.
I have however several comments and questions, that I would like to be addressed.
1) First I think a reference is missing. Indeed, to my knowledge, it is not the first work to use this kind of techniques. Cf [1]. This does not alterate however the novelty of the approach.
2) « We construct a linear DNN-based BoF » : I do not like this formulation. Here, you assume that you build a ResNet-50 with 1x1 as a representation and have a last final linear layer as a classifier. One could also claim it is a ResNet-48 as a representation followed by 2 layers of 1x1 as a classifier.
3) « our proposed model architecture is simpler » this is very subjective because for instance the FV models are learned in a layer-wise fashion, which makes their learning procedure more interpretable because each layer objective is specified. Furthermore, analyzing these models is now equivalent to analyze a cascade of fully connected layers, which is not simple at all.
4) Again, the interpretability mentioned in Sec. 3  is in term of spatial localization, not mapping. I think it is important to make clear this consideration. Indeed, this work basically leaves the problem of understanding general CNNs to the problem of understanding MLPs.
5) The graphic of the Appendix A is a bit misleading : it seems 13 downsampling are performed whereas it is not the case, because the first element of each group of block is actually only done once.(if I understood correctly)
6) I think the word feature is sometimes mis-used: sometimes it seems it can refer to a patch, sometimes to the code for a patch. (« Surprisingly, feature sizes assmall as 17 × 17 pixels »)
I got also few questions:
Q1 : I was wondering if you did try manifold learning on the patches ? Do you expect it to work ?
Q2 : Is there a batch normalization in the FC or a normalization? Did you try to threshold the heat maps before feeding them to the linear layer? I'm wondering indeed if the amplitude of those heatmaps is really key.
Q3 : do you think it would be easy to exploit the non-overlapping patches for a better parallelization of computations ?
Finally, I find very positive the amount of experiments to test the similarity with standard CNNs. Of course, it’s far from being a formal proof, but I think it is a very nice first step.
[1] Oyallon, Edouard, Eugene Belilovsky, and Sergey Zagoruyko. "Scaling the scattering transform: Deep hybrid networks." International Conference on Computer Vision (ICCV). 2017.


This paper presents a method to autonomously find periodicities in a signal. It is based on the same idea of using Fourier Transform and autocorrelation function presented in Vlachos et al. (2005). While showing interesting results this method does not perform well on noisy signals or signals with multiple periodicities. Thus, our method adds several new extra steps (hints clustering, filtering and detrending) to fix these issues. Experimental results show that the proposed method outperforms state of the art algorithms.

This paper focuses on the extraction of the (multi) periodicities from a signal. The paper describes the conventional method based on the Fourier transformation and/or autocorrelation methods, and proposed method, which first detects a distribution of spectral leakages, and prune the periodicity hints by using a clustering algorithm. The proposed method is also extended to deal with multi-periodicities. The effectiveness of the proposed method is shown with the controlled simulation data and several real data. This paper is well written (note it is over 8 pages though), but it is not learning-based approach, and would not best fit to major ICLR interests.
Comments:
- The abstract needs to be more self-consistent without referring the citation for a brief explanation. Also it should have more detailed experimental discussions.
- Algorithm 1 needs some refinement (too code-like, although it is understandable). For example, several methods (nextBinValue and append) would be better to be replaced with other (human readable) expressions.


REPRESENTING FORMAL LANGUAGES: A COMPARISON BETWEEN FINITE AUTOMATA AND RECURRENT NEURAL NETWORKS
We investigate the internal representations that a recurrent neural network (RNN) uses while learning to recognize a regular formal language. Specifically, we train a RNN on positive and negative examples from a regular language, and ask if there is a simple decoding function that maps states of this RNN to states of the minimal deterministic finite automaton (MDFA) for the language. Our experiments show that such a decoding function indeed exists, and that it maps states of the RNN not to MDFA states, but to states of an abstraction obtained by clustering small sets of MDFA states into "superstates". A qualitative analysis reveals that the abstraction often has a simple interpretation. Overall, the results suggest a strong structural relationship between internal representations used by RNNs and finite automata, and explain the well-known ability of RNNs to recognize formal grammatical structure.

Paper Summary -
The authors trained RNNs to recognize formal languages defined by random regular expressions, then measured the accuracy of decoders that predict states of the minimal deterministic finite automata (MDFA) from the RNN hidden states. They then perform a greedy search over partitions of the set of MDFA states to find the groups of states which, when merged into a single decoder target, maximize prediction accuracy. For both the MDFA and the merged classes prediction problems, linear decoders perform as well as non-linear decoders.
Clarity - The paper is very clear, both in its prose and maths.
Originality - I don't know of any prior work that approaches the relationship between RNNs and automata in quite this way.
Quality/Significance - I have one major concern about the interpretation of the experiments in this paper.
The paper seems to express the following logic:
1 - linear (and non-linear) decoders aren't so good at predicting MDFA states from RNN hidden states
2 - if we make an "abstract" finite automata (FA) by merging states of the MDFA to optimize decoder performance, the linear (and non-linear) decoders are much better at predicting this new, smaller FA's states.
3 - thus, trained RNNs implement something like an abstract FA to recognize formal languages.
However, a more appropriate interpretation of these experiments seems to be:
1 - (same)
2 - if we find the output classes the decoder is most often confused between, then merge them into one class, the decoder's performance increases -- trivially. in other words, you just removed the hardest parts of the classification problem, so performance increased. note: performance also increases because there are fewer classes in the merged-state FA prediction problem (e.g., chance accuracy is higher).
3 - thus, from these experiments it's hard to say much about the relationship between trained RNNs and finite automata.
I see that the "accuracy" measurement for the merged-state FA prediction problem, \rho, is somewhat more complicated than I would have expected; e.g., it takes into account \delta and f(h_t) as well as f(h_{t+1}). Ultimately, this formulation still asks whether any state in the merged state-set that contains f(h) transitions under the MDFA to the any state in the merged state-set that contains f(h_{t+1}). As a result, as far as I can tell the basic logic of the interpretation I laid out still applies.
Perhaps I've missed something -- I'll look forward to the author response which may alleviate my concern.
Pros - very clearly written, understanding trained RNNs is an important topic
Cons - the basic logic of the conclusion may be flawed (will await author response)
Minor -
The regular expression in Figure 6 (Top) is for phone numbers instead of emails.
"Average linear decoding accuracy as a function of M in the MDFA" -- I don't think "M" was ever defined. From contexts it looks like it's the number of nodes in the MDFA.
"Average ratio of coarseness" -- It would be nice to be explicit about what the "ratio of coarseness" is. I'm guessing it's (number of nodes in MDFA)/(number of nodes in abstracted DFA).
What are the integers and percentages inside the circles in Figure 6?
Figures 4 and 5 are difficult to interpret because the same (or at least very similar) colors are used multiple times.
I don't see "a" (as in a_t in the equations on page 3) defined anywhere. I think it's meant to indicate a symbol in the alphabet \Sigma. Maybe I missed it.


We propose a new application of embedding techniques to problem retrieval in adaptive tutoring. The objective is to retrieve problems similar in mathematical concepts. There are two challenges: First, like sentences, problems helpful to tutoring are never exactly the same in terms of the underlying concepts. Instead, good problems mix concepts in innovative ways, while still displaying continuity in their relationships. Second, it is difficult for humans to determine a similarity score consistent across a large enough training set. We propose a hierarchical problem embedding algorithm, called Prob2Vec, that consists of abstraction and embedding steps. Prob2Vec achieves 96.88% accuracy on a problem similarity test, in contrast to 75% from directly applying state-of-the-art sentence embedding methods. It is surprising that Prob2Vec is able to distinguish very fine-grained differences among problems, an ability humans need time and effort to acquire. In addition, the sub-problem of concept labeling with imbalanced training data set is interesting in its own right. It is a multi-label problem suffering from dimensionality explosion, which we propose ways to ameliorate. We propose the novel negative pre-training algorithm that dramatically reduces false negative and positive ratios for classification, using an imbalanced training data set.

This paper proposes a method for mathematical problem embedding, which firstly decomposes problems into concepts by an abstraction step and then trains a skip-gram model to learn concept embedding. A problem can be represented as the average concept (corresponding to those in the problem) embeddings. To handle the imbalanced dataset, a negative pre-training method is proposed to decrease false and false positives. Experimental results show that the proposed method works much better than baselines in similar problem detection, on an undergraduate probability data set.
Strong points:
(1)	The idea of decomposing problems into concepts is interesting and also makes sense.
(2)	The training method for imbalanced datasets is impressive.
Concerns or suggestions:
1.	The main idea of using contents to represent a problem is quite simple and straightforward. The contribution of this paper seems more on the training method for imbalanced data sets. But there are no comparisons between the proposed training method and previous related works. Actually, imbalance data sets are common in machine learning problems and there are many related works. The comparisons are also absent in experiments.
2.	The experimental data set is too small, with only 635 problems. It is difficult to judge the performance of the proposed model based on so small data set.
3.	The proposed method, which decomposes a problem into multiple concepts, looks general for many problem settings. For example, representing a movie or news article by tags or topics. In this way, the proposed method can be tested in a broader domain and on larger datasets.
4.	For the final purpose, comparing problem similarity, I am wondering what the result will be if we train a supervised model based problem-problem similarity labels?


In complex simulated environments, model-based reinforcement learning methods typically lag the asymptotic performance of model-free approaches. This paper uses two MuJoCo environments to understand this gap through a series of ablation experiments designed to separate the contributions of the dynamics model and planner. These reveal the importance of long planning horizons, beyond those typically used. A dynamics model that directly predicts distant states, based on current state and a long sequence of actions, is introduced. This avoids the need for many recursions during long-range planning, and thus is able to yield more accurate state estimates. These accurate predictions allow us to uncover the relationship between model accuracy and performance, and translate to higher task reward that matches or exceeds current state-of-the-art model-free approaches.

The paper proposes to use a multi-step prediction model in model-based RL. The proposed model maps from current state and a sequence of actions to the state after taking those actions. The paper demonstrates on 2 tasks that in a model-predictive control loop combined with planning by cross-entropy method, this can yield better asymptotic performance than using single-step models.
The insight of using multi-step prediction models is certainly appealing and makes a lot of sense in deterministic tasks. A systematic empirical comparison of multi-step deep models in RL is of interest, which this paper does provide to some extent.
An obvious limitation of the proposed deterministic multi-step forward model is the restriction to deterministic systems. One would expect that the performance deteriorates quickly as the system becomes more stochastic. An extension to the stochastic case along the lines of Chua et al, 2018 is non-trivial as capturing the stochasticity is typically more challenging in long-term predictions. Yet, the paper makes an additional assumption that is less clearly communicated: To be able to plan with a R-step model, one needs to be able to evaluate or approximate the sum of R rewards just from the first and last state in that R-long sequence. This work uses simply the reward at the end r(s_{t+R}) as a proxy which works well in these MuJoCo tasks but can fail horribly in others. One can imagine that a model not only outputs s_{t+R} but also the sum of R rewards given s_t and a_{t:t+R} which could work in more general settings but this is not explored in this paper. The contribution in this paper limited as the proposed approach as well as the experimental comparison is restricted to a relatively specific class of problems and no attempts to generalize are made.
The experiments nicely compare against using single-step dynamics models and the results show that using the multi-step models for MPC performs better in the two considered tasks. However, as fas as I understand both the ACP and Chua et al baseline using the single-step prediction accuracy to train their models. The paper is missing a comparison to single-step models that are trained using multi-step prediction losses ("backprop through time" as in Learning Nonlinear Dynamic Models by Langford et al 2009). These models should be much more robust to error blow-up for multi-step prediction and do not require the specific reward structure assumed in this paper.
The proposed R-step model-based RL approach could be connected to the use of options (the planner and model operate on R-step options, but the MPC does update the policy after every time step). It would be interesting to discuss this potential connection in the paper. The paper does a good job of discussing existing recent work in the deep RL literature but it would also be good to also discuss earlier work on multi-step prediction (e.g. in time-series modeling).
All in all, I think the paper makes a small contribution demonstrating that multi-step models are useful for model-based RL in specific domains -- which is interesting but certainly not surprising. Unfortunately the paper stops somewhat early by not comparing to relevant baselines (single-step models trained with multi-step losses) and by not considering tasks where the benefit of multi-step planning would be less clear.


FIXUP INITIALIZATION: RESIDUAL LEARNING WITHOUT NORMALIZATION
Normalization layers are a staple in state-of-the-art deep neural network architectures. They are widely believed to stabilize training, enable higher learning rate, accelerate convergence and improve generalization, though the reason for their effectiveness is still an active research topic. In this work, we challenge the commonly-held beliefs by showing that none of the perceived benefits is unique to normalization. Specifically, we propose fixed-update initialization (Fixup), an initialization motivated by solving the exploding and vanishing gradient problem at the beginning of training via properly rescaling a standard initialization. We find training residual networks with Fixup to be as stable as training with normalization -even for networks with 10,000 layers. Furthermore, with proper regularization, Fixup enables residual networks without normalization to achieve state-of-the-art performance in image classification and machine translation.

Summary:
A method is presented for initialization and normalization of deep residual networks. The method is based on interesting observations regarding forward and backward explosion in such networks with the standard Xavier or (He, 2015) initializations. Experiments with the new method show that it is able to learn with very deep networks, and that its performance is on a par with the best results obtained by other networks with more explicit normalization.
Advantages:
-	The paper includes interesting observations, resulting in two theorems,  which show the sensitivity of traditional initializations in residual networks
-	The method presented seems to work comparable to other state of the art initialization + normalization methods, providing overall strong empirical results.
Disadvantages:
-	The authors claim to suggest a method without normalization, but the claim is misleading: the network has additive and multiplicative normalization nodes, and their function and placement is at least as ‘mysterious’  as the role of normalization in methods like batch and layer normalization.
o	This significantly limits the novelty of the method: it is not ‘an intialization’ method, but a combination of initialization and normalization, which differ from previous ones in some details.
-	The method includes 3 components, of which only one is justified in a principled manner. The other components are not justified neither by an argument, nor by experiments. Without such experiments, it is not clear what actually works in this method, and what is not important.
-	The argument for the ‘justified’ component is not entirely clear to me. The main gist is fine, but important details are not explained so I could not get the entire argument step-by-step. This may be a clarity problem, or maybe indicate deeper problem of arbitrary decisions made without justification – I am not entirely sure. Such lack of clear argumentation occurs in several places
-	Experiments isolating the contribution of the method with respect to traditional initializations are missing (for example: experiments on Cifar10 and SVHN showing the result of traditional initializations with all the bells and whistles (cutout, mixup) as the zeroInit gets.
More detailed comments:
Page 3:
-	While I could follow the general argument before eq. 2, leading to the conclusion that the initial variance in a resnet explodes exponentially, I could not understand eq. 2. What is its justification and how is it related to the discussion before it? I think it requires some argumentation.
Page 4:
-	I did not understand example 2) for a p.h. set. I think an argument, reminder of the details of resnet, or a figure are required.
-	I could not follow the details of the argument leading to the zeroInit method:
o	How is the second design principle “Var[F_l(x_l)] = O( 1/L) justified?
As far as I can see, having Var[F_l(x_l)] = 1/L will lead to output variance of (1+1/L)^L =~e, which is indeed O(1). Is this the argument? Is yes, why wasn’t it stated? Also: why not smaller than O(1/L)?
o	Following this design principle several unclear sentences are stated:
	We strive to make Var[F_l(x_l)] = 1/L, yet we set the last convolutional layer in the branch to 0 weights. Does not it set Var[F_l(x_l)] = 0, in contradiction to the 1/L requirement?
	 “Assuming the error signal passing to the branch is O(1),” – what does the term “error signal” refer to? How is it defined? Do you refer to the branch’s input?
	I understand why the input to the m-th layer in the branch is O(\Lambda^m-1) if the branch input is O(1) but why is it claimed that “the overall scaling of the residual branch after update is O(\lambda^(2m-2))”? what is ‘the overall scaling after update’ (definition) and why is it the square of forward scaling?
-	The zero Init procedure step 3 is not justified by any argument in the proceeding discussion. Is there any reason for this policy? Or was it found by trial and error and is currently unjustified theoretically (justified empirically instead). This issue should be clearly elaborated in the text. Note that the addition of trainable additive and multiplicative elements is inserting the normalization back, while it was claimed to be eliminated. If I understand correctly, the ‘zeroInit’ method is hence not based on initialization (or at least: not only on initialization), but on another form of normalization, which is not more justified than its competitors (in fact it is even more mysterious: what should we need an additive bias before every element in the network?)
Page 5:
-	What is \sqrt(1/2) scaling? It should be defined or given a reference.
Page 6:
-	It is not stated on what data set figure 2 was generated.
-	In table 2, for Cifar-10 the comparison between Xavier init and zeroInit shows only a small advantage for the latter. For SVHN such an experiment is completely missing, and should be added.
o	It raises the suspect the the good results obtained with zeroInit in this table are only due to the CutOut and mixup used, that is: maybe such results could be obtained with CutOut+Mixup without zero init, using plain Xavier init? experiments clarifying this point are also missing.
Additional missing experiments:
-	It seems that  ZeroInit includes 3 ingredients (according to the box in page 4), among which only one (number 2) is roughly justified from the discussion.  Step 1) of zeroing the last layer in each branch is not justified –why are we zeroing the last layer and not the first, for example? Step 3 is not even discussed in the text – it appear without any argumentation. For such steps, empirical evidence should be brought, and experiments doing this are missing. Specifically experiments of interest are:
o	Using zero init without its step 3: does it work? The theory says it should.
o	Using only step 3 without steps 1,2. Maybe only the normalization is doing the magic?
The paper is longer than 8 pages.
I have read the rebuttal.
Regarding normalization: I think that there are at least two reasonable meanings to the word 'normalziation': in the wider sense is just means mechanism for reducing a global constant (additive normalization) and dividing by a global constant (multiplicative normalization). In this sense the constant parameters can be learnt in any way. In the narrow sense the constants have to be statistics of the data. I agree with the authors that their method is not normalization in sense 2, only in sense 1. Note that keeping the normalization in sense 1 is not trivial (why do we need these normalization operations? at least for the multiplicative ones, the network has the same expressive power without them).  I think the meaning of normalization  should be clearly explained in the claim for 'no  normalization'.
Regarding additional mathematical and empirical justifications required: I think such justifications are missing in the current paper version and are not minor or easy to add. I believe the work should be re-judged after re-submission of a version addressing the problems.


LEARNING RECURRENT BINARY/TERNARY WEIGHTS
Recurrent neural networks (RNNs) have shown excellent performance in processing sequence data. However, they are both complex and memory intensive due to their recursive nature. These limitations make RNNs difficult to embed on mobile devices requiring real-time processes with limited hardware resources. To address the above issues, we introduce a method that can learn binary and ternary weights during the training phase to facilitate hardware implementations of RNNs. As a result, using this approach replaces all multiply-accumulate operations by simple accumulations, bringing significant benefits to custom hardware in terms of silicon area and power consumption. On the software side, we evaluate the performance (in terms of accuracy) of our method using long short-term memories (LSTMs) and gated recurrent units (GRUs) on various sequential models including sequence classification and language modeling. We demonstrate that our method achieves competitive results on the aforementioned tasks while using binary/ternary weights during the runtime. On the hardware side, we present custom hardware for accelerating the recurrent computations of LSTMs with binary/ternary weights. Ultimately, we show that LSTMs with binary/ternary weights can achieve up to 12× memory saving and 10× inference speedup compared to the full-precision hardware implementation design.

* Summary
This paper proposes batch normalization for learning RNNs with binary or ternary weights instead of full-precision weights. Experiments are carried out on character-level and word-level language modeling, as well as sequential MNIST and question answering.
* Strengths
- I liked the variety of tasks used evaluations (sequential MNIST, language modeling, question answering).
- Encouraging results on specialized hardware implementation.
* Weaknesses
- Using batch normalization on existing binarization/ternarization techniques is a bit of an incremental contribution.
- All test perplexities for word-level language models in table 3 underperform compared to current vanilla LSTMs for that task (see Table 4 in https://arxiv.org/pdf/1707.05589.pdf), suggesting that the baseline LSTM used in this paper is not strong enough.
- Results on question answering are not convincing -- BinaryConnect has the same size while achieving substantially higher accuracy (94.66% vs 40.78%). This is nowhere discussed and the paper's major claims "binaryconnect method fails" and "our method [...] outperforms all the existing quantization methods" seem unfounded (Section 5.5).
- In the introduction, I am lacking a distinction between improvements w.r.t. training vs inference time. As far as I understand, quantization methods only help at reducing memory footprint or computation time during inference/test but not during training. This should be clarified.
- In the introduction on page 2 is argued that the proposed method "eliminates the need for multiplications" -- I do not see how this is possible. Maybe what you meant is that it eliminates the need for full-precision multiplications by replacing them with multiplications with binary/ternary matrices?
- The notation is quite confusing. For starters, in Section 2 you mention "a fixed scaling factor A" and I would encourage you to indicate scalars by lower-case letters, vectors by boldface lower-case letters and matrices by boldface upper-case letters. Moreover, it is unclear when calculations are approximate. For instance, in Eq. 1 I believe you need to replace "=" with "\approx". Likewise for the equation in the next to last line on page 2. Lastly, while Eq. 2 seems to be a common way to write down LSTM equations, it is abusive notation.
* Minor Comments
- Abstract: What is ASIC? It is not referenced in Section 6.
- Introduction: What is the justification for calling RNNs over-parameterized? This seems to depend on the task.
- Introduction; contributions: Here, I would like to see a distinction between gains during training vs test time.
- Section 3.2 comes out of nowhere. You might want to already mention why are introducing batch normalization at this point.
- The boldfacing in Table 1, 2 and 3 is misleading. I understand this is done to highlight the proposed method, but I think commonly boldfacing is used to highlight the best results.
- Figure 2b. What is your hypothesis why BPC actually goes down the longer the sequence is?
- Algorithm 1, line 14: Using the cross-entropy is a specific choice dependent on the task. My understanding is your approach can work with any differentiable downstream loss?


Convolutional neural networks memorize part of their training data, which is why strategies such as data augmentation and drop-out are employed to mitigate overfitting. This paper considers the related question of "membership inference", where the goal is to determine if an image was used during training. We consider membership tests over either ensembles of samples or individual samples.
First, we show how to detect if a dataset was used to train a model, and in particular whether some validation images were used at train time. Then, we introduce a new approach to infer membership when a few of the top layers are not available or have been fine-tuned, and show that lower layers still carry information about the training samples. To support our findings, we conduct large-scale experiments on Imagenet and subsets of YFCC-100M with modern architectures such as VGG and Resnet.

==============Final Evaluation================
I have gone through the other reviews as well as the author response.
Firstly, I would like to thank the authors for providing detailed responses to my questions.
In general, I agree with R2 that the paper generally has some potentially interesting ideas and results but the manner in which the current draft is organized and presented makes it hard to grasp them and there is a lack of coherent message about what the paper is about.
Moreover, from my understanding the analysis in David McKay’s book (Chapter 41) concerns a single neuron (and the number of parameters for a single neuron). As pointed out by R2, with depth there are a lot more number of possible ways in which one could carve out decision boundaries to separate data points, thus, it is not clear that the loose linear upper bound holds Specifically, as one might expect with depth it could be possible that linear capacity increase is a lower bound (I am not suggesting that it is, but that possibility should be considered and explained in the paper). Similarly, it would be good to formally connect the capacity to the rate of memorization before making a statement about them being related (as suggested in the initial review). In general, I feel this section could use some tighter formalism and justifications.
I also remain unconvinced by the response to my issue with the claim “Our experiments show that our networks can remember a large number of images and distinguish them from unseen images”, where the negative images are also seen by the memorization model, so they are not unseen. The authors address this by saying 3M of the 15 M negatives have been seen. That does not seem like a small enough percentage to claim that these are “unseen” images.
In general, I feel the paper is interesting but would benefit from a major revision which makes the message of the paper more clear, and addresses these and other issues raised in the review phase. Thus I am holding my current rating.
==================
Summary
The paper trains classification models to classify a labeling of a subset of images (assigned with label 1) from the rest of the images (assigned with a label 0). Firstly, the paper shows that deep learning models are able to learn such classifiers and get low training loss. It then proposes to use this model to ``attack’’ task-specific models to perform membership inference, i.e. figuring out if an image provided in a set was used in training or not.
Strengths
+ The paper thoroughly covers related work and provides context.
+ Results on confidence as a signature of a dataset are interesting.
Weaknesses
[Motivation]
1. In general, recent work has found that the raw number of parameters has little to do with the size of the model class or the capacity of a model for deep models, and thus work like [A] has been trying to come up with better complexity measures for models to explain generalization. Thus, without sufficient justification the assertion in the paper that the capacity of the network is well approximated by the number of parameters does not seem correct. Also, the claim in Fig. 1 that the transition from ‘’high capacity’’ to low capacity happens at the number of parameters in the network seems a bit loose and hard to substantiate from what I understand, and should be toned down. (*)
[Capacity]
2. Sec. 3.3, Fig. 3: The capacity (in terms of parameters)of both Resnet-18 and VGG-16 is higher than the capcity for YFCC100M dataset for n=10K images (comes to 161K bits), while the capacity of Resnet-18, with 14.7 million parameters (assuming float32 encoding) has 14.7 * 32 bits = 470.4 million bits, thus capacity alone cannot explain why VGG converges faster than Resnet-18, since both networks exceed the capacity, and capacity does not seem to have an established formal connection to rate of memorization. This is something which would need to be explained/ substantiated separately. (*)
3. Scenario discussed in Sec. 4 seems somewhat impractical. Given a set of m images, it is not clear that a classifier that is trained to detect between train and validation is sufficient, as one might also need to figure out if it is neither train nor val, which is a very practical scenario.
4. Fig. 3 (right): It is not clear why the fact that the classifier is able to predict which dataset the image ‘m’ corresponds to is useful or practical, as this seems to be a property of the set ‘m’ rather than the property of the trained classification model (f_\theta). Please clarify. On the other hand it is clear that using the confidence of the model to predict the dataset is a useful property, but the right side of the Fig. is very confusing. (*)
6. It is not clear to me what the point of Sec. 5 is, given a trained model, one wants to figure out if an image was present in the training of the model. While the baseline approaches seem to make use of the model confidence, I cannot see how the proposed approach (which uses a classifier) makes use of the original model. It is also not clear why Table. 3 does not report the Bayes baseline results. Also, does this section use the classifier for predicting the dataset, or is the approach reported in the section, the MAT approach?
7. ``Our experiments show that our networks can remember a large number of images and distinguish them from unseen images’’ -- this does not seem to be true, since the model is trained on both n as well as N -n ``unseen’’ images which it labels as the negative class, thus the negative class is also seen by the memorization model. (*)
Minor Points
1. It is not clear that training a network to classify a set from another set is necessarily equivalent to ``memorization’’. In addition, the paper would also need to show that such a model does not generalize to a validation set of images. This is probably obvious given the results from Zhang et.al. but should be included as a sanity check.
2. Figure 3: it is confusing to call the cumulative distribution of the maximum classification score as the CDF of the model (y-axis fig. 3 left) as CDF means something else generally in such contexts, as the CDF of a predictor.
References:
[A]: Blier, Léonard, and Yann Ollivier. 2018. ``The Description Length of Deep Learning Models.’’ arXiv [cs.LG]. arXiv. http://arxiv.org/abs/1802.07044.
Preliminary Evaluation
There are numerous issues with the writing and clarity of the paper, while it seems like some of the observations around the confidence of classifiers are interesting, in general the connection between those set of results and the ``memorization’’ capabilities of the classifier trained to remember train vs val images is not clear in general. Important points for the rebuttal are marked with (*).


High-dimensional sparse reward tasks present major challenges for reinforcement learning agents. In this work we use imitation learning to address two of these challenges: how to learn a useful representation of the world e.g. from pixels, and how to explore efficiently given the rarity of a reward signal? We show that adversarial imitation can work well even in a high dimensional observation space. Surprisingly, the discriminator network, providing a learned reward function, can be tiny, comprising as few as 128 parameters, and can be easily trained using the most basic GAN formulation. Our approach removes limitations present in most contemporary imitation approaches: requiring no demonstrator actions (only video), no special initial conditions or warm starts, and no explicit tracking of any single demo. The proposed agent can solve a challenging robot manipulation task of block stacking from only video demonstrations and sparse reward, in which the non-imitating agents fail to learn completely. Furthermore, our agent learns much faster than competing approaches that depend on hand-crafted, staged dense reward functions, and also better compared to standard GAIL baselines. Finally, we develop a new adversarial goal recognizer that in some cases allows the agent to learn stacking without any task reward, purely from imitation.
minute differences between the agent and expert for a deep neural network discriminator to exploit. Furthermore, optimizing deep adversarial networks is notoriously difficult, although there is progress towards stabilizing this problem; see e.g. ; .
In this work, we show that if GAIL is provided the right kind of features, it can actually easily handle high-dimensional pixel observations using a single layer discriminator network. We can also improve its efficiency in terms of environment interactions by using a Deep Distributed Deterministic Policy Gradients (D4PG) agent , which is a state-of-the-art off-policy method for control, that can take advantage of a replay buffer to store past experiences.
We show that several types of features can be used successfully with a tiny, single-layer adversary, in cases where a deep adversary on pixels would fail completely. Specifically: self-supervised embeddings, e.g. Contrastive Predictive Coding (van den Oord et al., 2018); surprisingly, random projections through a deep residual network; and value network features from the D4PG agent itself. Concurrently with (Kostrikov et al., 2018), we additionally show how to modify GAIL for the off-policy case of D4PG agents with experience replay.
In our experiments, we demonstrate that our proposed approach is able to, from pixels, solve a challenging simulated robotic block stacking task using only demonstrations and a sparse binary reward indicating whether or not the stack has completed. Previous imitation approaches to this task have all used dense, staged task rewards crafted by humans and/or true state instead of pixels. In addition to reducing the dependency on hand-crafted rewards, our approach learns to stack faster than the dense staged reward baseline agent with the same amount of actor processes.
The main contributions of this paper are the following:
• A 6-DoF Jaco robot arm agent that learns block stacking from demonstration videos and sparse reward. On a simulated Jaco arm it achieves 94% success rate, compared to ∼ 29% using a behavior cloning agent with an equivalent number of demonstrations.
• An adversary-based early termination method for actor processes, that improves task performance and learning speed by creating a natural curriculum for the agent.
• An agent that learns with no task reward using an auxiliary goal recognizer adversary, achieving 55% stacking success from video imitation only.
• Ablation experiments on Jaco stacking as well as a 2D planar walker benchmark (Tassa et al., 2018), to understand the specific reasons for improvement in our agent. We find that random projections with a linear discriminator work surprisingly well in some cases, and that using value network features is even better.

The submission describes a sort of hybrid between reinforcement learning and imitation learning, where an auxiliary imitation learning objective helps to guide the RL policy given expert demonstrations.  The method consists of concurrently maximizing an RL objective--augmented with the GAIL discriminator as a reward—and minimizing the GAIL objective, which optimizes the discriminator between expert and policy-generated states.  Only expert states (not actions) are required, which allows the method to work given only videos of the expert demonstrations.  Experiments show that adding the visual imitation learning component allows RL to work with sparse rewards for complex tasks, in situations where RL without the imitation learning component fails.
Pros:
+ It is an interesting result that adding a weak visual imitation loss dramatically improves RL with sparse rewards
+ The idea of a visual imitation signal is well-motivated and could be used to solve practical problems
+ The method enables an ‘early termination’ heuristic based on the imitation loss, which seems like a nice heuristic to speed up RL in practice
Cons:
+ It seems possible that imitation only helps RL where imitation alone works pretty well already
+ Some contributions are a bit muddled: e.g., the “learning with no task reward” section is a little confusing, because it seems to describe what is essentially a variant of normal GAIL
+ The presentation borders on hand-wavy at parts and may benefit from a clean, formal description
The submission tackles a real, well-motivated problem that would appeal to many in the ICLR community.  The setting is attractive because expert demonstrations are available for many problems, so it seems obvious that they should be leveraged to solve RL problems—especially the hardest problems, which feature very sparse reward signals.  It is an interesting observation that an imitation loss can be used as  a dense reward signal to supplement the sparse RL reward.  The experimental results also seem very promising, as the imitation loss seems to mean the difference between sparse-reward RL completely failing and succeeding.  Some architectural / feature selection details developed here seem to also be a meaningful contribution, as these factors also seem to determine the success or failure of the method.
My biggest doubt about the method is whether it really only works where imitation learning works pretty well already.  If we don’t have enough expert examples for imitation learning to work, or if the expert is not optimizing the given reward function, then it is possible that adding the imitation loss is detrimental, because it induces an undesirable bias.  If, on the other hand, we do have enough training examples for imitation learning to succeed and the expert is optimizing the given reward function, then perhaps we should just do imitation learning instead of RL.  So, it is possible that there is some sweet spot where this method makes sense, but the extent of that sweet spot is unclear to me.
The experiments are unclear on this issue for a few reasons.  First, figure 4 is confusing, as it is titled ‘comparison to standard GAIL', which makes it sound like a comparison to standard imitation learning.  However, I believe this figure is actually showing the performance of different variants of GAIL used as a subroutine in the hybrid RL-IL method.  I would like to know how much reward vanilla GAIL (without sparse rewards) achieves in this setting.  Second, figure 8 seems to confirm that some variant of vanilla imitation learning (without sparse rewards) actually does work most of the time, achieving results that are as good as some variants of the hybrid RL-IL method.  I think it would be useful to know, essentially, how much gain the hybrid method achieves over vanilla IL in different situations.
Another disappointing aspect of the paper is the ‘learning with no task reward’ section, which is a bit confusing.  The concept seems reasonable at a first glance, except that once we replace the sparse task reward with another discriminator, aren’t we firmly back in the imitation learning setting again?  So, the motivation for this section just seems a bit unclear to me.  This seems to be describing a variant of GAIL with D4PG for the outer optimization instead of TRPO, which seems like a tangent from the main idea of the paper.  I don’t think it is necessarily a bad idea to have another discriminator for the goal, but this part seems somewhat out of place.
On presentation: I think the presentation is a bit overly hand-wavy in parts.  I think the manuscript could benefit from having a concise, formal description.  Currently, the paper feels like a series of disjoint equations with unclear connections among them.  The paper is still intelligible, but not without knowing a lot of context relating to RL/IL methods that are trendy right now.  I feel that this is an unfortunate trend recently that should be corrected.  Also, I’m not sure it is really necessary to invoke “GAIL” to describe the IL component, since the discriminator is in fact linear, and the entropy component is dropped.  I think “apprenticeship learning” may be a more apt analogy.
On originality: as far as I can tell, the main idea of the work is novel.  The work consists mainly of combining existing methods (D4PG, GAIL) in a novel way.  However, some minor novel variations of GAIL are also proposed, as well as novel architectural considerations.
Overall, this is a nice idea applied to a well-motivated problem with promising results, although the exact regime in which the method succeeds could be better characterized.


In many applications we seek to optimize an expectation with respect to a distribution over discrete variables. Estimating gradients of such objectives with respect to the distribution parameters is a challenging problem. We analyze existing solutions including finite-difference (FD) estimators and continuous relaxation (CR) estimators in terms of bias and variance. We show that the commonly used Gumbel-Softmax estimator is biased and propose a simple method to reduce it. We also derive a simpler piece-wise linear continuous relaxation that also possesses reduced bias. We demonstrate empirically that reduced bias leads to a better performance in variational inference and on binary optimization tasks.

The papers studies estimators of gradients taken from expectations with respect to the distribution parameters. The paper has studied two main types of estimators, Finite Difference and Continuous Relaxation. The paper made several improvements to existing estimators.
My rating of the paper in different aspects (quality 6, clarity 8, originality 6, significance 4).
Pros:
1. The paper has made a nice introduction of FD and CR estimators. The improvements over previous estimators are concrete -- it is generally clear to see the benefit of these improvements.
2. The first method reduces the running time of the RAM estimator. The second method (IGM) reduces the bias of GM estimator. The first improvement avoids many function evaluations when the probability is extreme. The second improvement helps to correct bias introduced by continuous approximation of \zeta_i itself.
Cons:
1. the paper content is a little disjointed: the improvement over RAM has not much relation with later improvements. It seems the paper is stacking different things into the paper.
2. All these improvements are not very significant considering a few previous papers on this topic. Some arguments are not rigorous. (see details below)
3. A few important papers are not well discussed and omitted from the experiment section.
Detailed comments
1. The REBAR estimator [Tucker et al., 2017] and the LAX estimator [Grathwohl et al., 2018] use continuous approximation and correct it to be unbiased. These papers in this thread are not well discussed in the paper. They are not compared in the experiment either.
2. In the equation 7 and above: what does 4 mean? When beta \neq 4, do you still get unbiased estimation? My understanding is that the estimator is unbiased only when beta=4. (correct me if I'm wrong)
3. The paper argues that the variance of the estimator is mostly decided by the variance of q(zeta)^-1 when the function is smooth. I feel this argument is not very clear. First, what do you mean by saying the function is smooth? The derivative is near a constant in [0, 1]?
4. In the PWL development, the paper argues that we can choose alpha_i \approx 1/(q_i(1-q_i)) to minimize the variance. However, my understanding is, the smaller alpha_i, the smaller variance.


Knowledge Graph Embedding (KGE) is the task of jointly learning entity and relation embeddings for a given knowledge graph. Existing methods for learning KGEs can be seen as a two-stage process where (a) entities and relations in the knowledge graph are represented using some linear algebraic structures (embeddings), and (b) a scoring function is defined that evaluates the strength of a relation that holds between two entities using the corresponding relation and entity embeddings. Unfortunately, prior proposals for the scoring functions in the first step have been heuristically motivated, and it is unclear as to how the scoring functions in KGEs relate to the generation process of the underlying knowledge graph. To address this issue, we propose a generative account of the KGE learning task. Specifically, given a knowledge graph represented by a set of relational triples (h, R, t), where the semantic relation R holds between the two entities h (head) and t (tail), we extend the random walk model (Arora et al., 2016a) of word embeddings to KGE. We derive a theoretical relationship between the joint probability p(h, R, t) and the embeddings of h, R and t. Moreover, we show that marginal loss minimisation, a popular objective used by much prior work in KGE, follows naturally from the log-likelihood ratio maximisation under the probabilities estimated from the KGEs according to our theoretical relationship. We propose a learning objective motivated by the theoretical analysis to learn KGEs from a given knowledge graph. The KGEs learnt by our proposed method obtain state-of-the-art performance on FB15K237 and WN18RR benchmark datasets, providing empirical evidence in support of the theory.

This paper proposes to perform the link prediction in knowledge bases by introducing a new scoring function and theoretically motivating their method. The authors validate their proposed approach through several experiments.
This paper reads well and the results appear sound. I personally find the theoretical argument behind the proposed scoring function very interesting. Unfortunately, the contribution seems rather small to be accepted for ICLR. This is a straight application and combination of existing pieces with not much originality and without being backed up by very strong experimental results. My concerns are as follows:
   - Having only results on two flawed datasets (considering the inverse relations in them) makes it hard to evaluate the quality of the method. I suggest conducting the experiments on the FB15K-237 and WN18RR from [1] instead.
   - You only evaluate on MR and Hits@10, but it is standard to include metrics like MRR and Hits@1 and 3 also, since no metric is perfect for this task.
   - Since the goal of the work is not providing a state of the art method, and focus on the theoretical understanding of their scoring function, it is of high importance to assess the characteristics of their embeddings and scoring function through designing other experiments. As a result, I suggest to study the geometric behavior of their embeddings and compare it to the other methods. Further, investigating the semantic purity of the embeddings by calculating the entropy of the type distribution of the entities, similar as [2], can shed more light on the significance of their method.
On overall, although the proposed method seems a direct application of Arora et al.,2016a, I find their extension novel and quite interesting, But the paper needs more experimental results to validate the idea.
[1] Dettmers, Tim, et al. "Convolutional 2d knowledge graph embeddings.", AAAI-18.
[2] Ding, Boyang, et al. "Improving Knowledge Graph Embedding Using Simple Constraints.", ACL-18.


The sophisticated structure of Convolutional Neural Network (CNN) allows for outstanding performance, but at the cost of intensive computation. As significant redundancies inevitably present in such a structure, many works have been proposed to prune the convolutional filters for computation cost reduction. Although extremely effective, most works are based only on quantitative characteristics of the convolutional filters, and highly overlook the qualitative interpretation of individual filter's specific functionality. In this work, we interpreted the functionality and redundancy of the convolutional filters from different perspectives, and proposed a functionality-oriented filter pruning method. With extensive experiment results, we proved the convolutional filters' qualitative significance regardless of magnitude, demonstrated significant neural network redundancy due to repetitive filter functions, and analyzed the filter functionality defection under inappropriate retraining process. Such an interpretable pruning approach not only offers outstanding computation cost optimization over previous filter pruning methods, but also interprets filter pruning process.

In this paper, the authors propose a method for pruning the convolutional filters. This method first separates the filters into clusters based on similarities defined with both Activation Maximization (AM) and back-propagation gradients. Then pruning is conducted based on the clustering results, and the contribution index that is calculated based on backward-propagation gradients. The proposed method is compared with a baseline method in the experiments.
I consider the proposed method as novel, since I do not know any filter pruning methods that adopt a similar strategy. Based on my understanding of the proposed method, it might be useful in convolutional filter pruning.
It seems that "interpretable" might not be the most proper word to summarize the method. It looks like that the key concept of this paper, including smilarity defined in Equation (3), and the contribution index defined in Equation (7) are not directly relevant to interpretability. Therefore, I would consider change the title of the paper, for example, to "Convolutional Filter Pruning Based on Functionality ".
In terms of writing, I have difficulty understanding some details about the method.
In filter clustering, how can one run k-means based on pair-wise similarity matrix $S_D$? Do you run kernel k-means, or you  apply PCA to $S_D$ before k-means? What is the criterion of choosing the number of clusters in the process of grid search?
Are filter level pruning, are cluster level pruning and layer level pruning three pruning strategies in the algorithm? It seems to me that you just apply one pruning strategy based on the clusters and contribution index, as shown in Figure 3.
In the subsubsection "Cluster Level Pruning", by "cluster volume size", denoted with$length(C^l_c)$, do you mean the size of cluster, i.e., the number of elements in each cluster? This is the first time I see the term "volume size". I assume the adaptive pruning rate, denoted by $R_{clt}^{(c,l)}$, is a fraction. But it looks to me that $length(C^l_c)$ is an integer. So how can it be true that $R_{clt}^{(c,l)} = length(C^l_c)$?
In the subsubsection "Layer Level Pruning", how is the value of $r$ determined?
The authors have conducted several experiments. These experiments help me understand the advantages of the proposed method. However, in the experiments, the proposed method is compared to only one baseline method. In recent years, a large number of convolutional filter pruning methods have been proposed, as mentioned in the related work section. I am not convinced that the proposed method is one of the best methods among all these existing methods. I would suggest the authors provide more experimental comparison, or explain why comparing with these existing methods is irrelevant.
Since the proposed method is heuristic, I would also like the authors to illustrate that each component of the method is important, via experiment. How would the performance of the proposed method be affected, if we define the similarity $S_D$ in Equation (3) using only $V$ or $\gamma$, rather than both $V$ and $\gamma$? How would the performance of the proposed method be affected, if we prune randomly, rather than prune based on the contribution index?
In summary, I think the method proposed in this paper might be reasonable. But I do not suggest acceptance, unless the author can improve the writing and include more experimental results.