Automatic classification of objects is one of the most important tasks in engineering and data mining applications. Although using more complex and advanced classifiers can help to improve the accuracy of classification systems, it can be done by analyzing data sets and their features for a particular problem. Feature combination is the one which can improve the quality of the features. In this paper, a structure similar to Feed-Forward Neural Network (FFNN) is used to generate an optimized linear or non-linear combination of features for classification. Genetic Algorithm (GA) is applied to update weights and biases. Since nature of data sets and their features impact on the effectiveness of combination and classification system, linear and non-linear activation functions (or transfer function) are used to achieve more reliable system. Experiments of several UCI data sets and using minimum distance classifier as a simple classifier indicate that proposed linear and non-linear intelligent FFNN-based feature combination can present more reliable and promising results. By using such a feature combination method, there is no need to use more powerful and complex classifier anymore.

This paper proposes using a feedforward neural network (FFNN) to extract intermediate features which are input to a 1NN classifier. The parameters of the FFNN are updated via a genetic algorithm with a fitness function defined as the error on the downstream classification, on a held-out set. The performance of this approach is measured on several UCI datasets and compared with baselines.
– The paper’s main contribution seems to be a neural network with a GA optimization for classification that can learn “intelligent combinations of features”, which can be easily classified by a simple 1NN classifier. But isn't this exactly what neural networks do – learn intelligent combinations of features optimized (in this case, via GA) for a downstream task? This has already been successfully applied in multiple domains eg. in computer vision (Krizhevsky et al, NIPS 2011), NLP (Bahdanau et al 2014), image retrieval (Krizhevsky et al. ESANN 2011) etc, and also studied comprehensively in autoencoding literature. There also exists prior work on optimizing neural nets via GA (Leung, Frank Hung-Fat et al., IEEE Transactions on Neural networks 2003). However, this paper claims both as novelties while not offering any improvement / comparison.
– The claim “there is no need to use more powerful and complex classifier anymore” is unsubstantiated, as the paper’s approach still entails using a complex classifier (a FFNN) to learn an optimal intermediate representation.
– The choice of activations is not motivated, and performance on variants is not reported. For instance, why is that particular sigmoid formulation used?
– The use for a genetic algorithm for optimization is not motivated, and no comparison is made to the performance and efficiency of other approaches (like standard backpropagation). So it is unclear why GA makes for a better choice of optimization, if at all.
– The primary baselines compared to are unsupervised methods (PCA and LDA), and so demonstrating improvements over those with a supervised representation does not seem significant or surprising. It would be useful to compare with a simple neural network baseline trained for K-way classification with standard backpropagation (though the UCI datasets may potentially be too small to achieve good performance).
– The paper is poorly written, containing several typos and incomplete, unintelligible sentences, incorrect captions (eg. Table 4) etc.


Off-policy learning, the task of evaluating and improving policies using historic data collected from a logging policy, is important because on-policy evaluation is usually expensive and has adverse impacts. One of the major challenge of offpolicy learning is to derive counterfactual estimators that also has low variance and thus low generalization error. In this work, inspired by learning bounds for importance sampling problems, we present a new counterfactual learning principle for off-policy learning with bandit feedbacks. Our method regularizes the generalization error by minimizing the distribution divergence between the logging policy and the new policy, and removes the need for iterating through all training samples to compute sample variance regularization in prior work. With neural network policies, our end-to-end training algorithms using variational divergence minimization showed significant improvement over conventional baseline algorithms and is also consistent with our theoretical results.

In this paper the authors studied the problem of off-policy learning, in the bandit setting when a batch log of data generated by the baseline policy is given. Here they first summarize the surrogate objective functions derived by existing approaches such as importance sampling and variance regularization (Swaminathan et. al). Then they extend the results in Theorem 2 of the paper by Cortes et. al (which also uses the empirical Bernstein inequality by Maurer and Pontil), and derive a new surrogate objective function that involves the chi-square divergence. Furthermore, the authors also show that the lower bound of this objective function can be iteratively approximated by variational f-GAN techniques, which could potentially be more numerically stable and empirically has lower variance.
In general, I think the problem studied in this paper is very interesting, and the topic of counterfactual learning, especially policy optimization with the use of offline and off-policy log data, is important. However, I think the theoretical contribution in this paper on off-policy learning is quite incremental. Also the parts that involve f-GAN is still questionable to me.
Detailed comments:
In these variance regularization formulations (for example the one proposed in this paper, or the one derived in Swaminathan's paper), \lambda can be seen as a regularization parameter that trades-off bias and variance of the off-policy value estimator R(h) (for example the RHS of equation 6). To exactly calculate \lambda either requires the size of the policy class (when the policy class is finite), or the complexity constants (which exists in C_1 and C_2 in equation 7, but it is not clearly defined in this paper). Then the main question is on how to choose \lambda such that the surrogate objective function is reasonable. For example in the safety setting (off-policy policy learning with baseline performance guarantees, for example see the problem setting in the paper by P. Thomas 2015: High Confidence off-policy improvement), one always needs the upper-bound in 6) to hold. This makes the choice of \lambda crucial and challenging. Unfortunately I don't see much discussions in this paper about choosing \lambda, even in the context of bias-variance trade-offs. This makes me uncomfortable in believing that the results in experiments hold for other (reasonable) choices of \lambda.
The contribution of this paper is of two-fold: 1) the authors extend the results from Cortes's paper to derive a new surrogate objective function, and 2) they show how this objective can be approximated by f-GAN techniques. The first contribution is rather incremental as it's just a direct application of Theorem 2 in Cortes's paper. Regarding the second contribution, I am a bit concerned about the derivations of Equation 9, especially the first inequality and the second equality. I see that the first inequality is potentially an application of the conjugate function inequality, but more details are needed (f^* is not even defined). For the second equality, it's unclear to me how one can swap the sup and the E_x operators. More explanations are definitely needed to show their mathematical correctness, especially when this part is a main contribution.  Even if the derivations are right, the f-GAN surrogate objective is a lower bound of the surrogate objective function, while the surrogate function is an upper bound of the true objective function (which is inaccessible). How does one guarantees that the f-GAN surrogate objective is a reasonable one?
Numerical comparisons between the proposed approach, and the approach from Swaminathan's paper are required to demonstrate the superiority of the proposed approach. Are there comparisons in performance between the approach from the original chi-square surrogate function and the one from the f-GAN objective (in order to showcase the need of using f-GAN) as well?
Minor comments:
In experimental section, method POEM is not defined.
The paper is in an okay status. But there are several minor typos, for example \hat{R}_{(} in page 3, and several typos in Algorithm 1 and Algorithm 2.
In general, I think this paper is studying an interesting topic, but the aforementioned issues make me feel that the paper's current status is still unsuitable for publication.


The Wasserstein probability metric has received much attention from the machine learning community. Unlike the Kullback-Leibler divergence, which strictly measures change in probability, the Wasserstein metric reflects the underlying geometry between outcomes. The value of being sensitive to this geometry has been demonstrated, among others, in ordinal regression and generative modelling, and most recently in reinforcement learning. In this paper we describe three natural properties of probability divergences that we believe reflect requirements from machine learning: sum invariance, scale sensitivity, and unbiased sample gradients. The Wasserstein metric possesses the first two properties but, unlike the Kullback-Leibler divergence, does not possess the third. We provide empirical evidence suggesting this is a serious issue in practice. Leveraging insights from probabilistic forecasting we propose an alternative to the Wasserstein metric, the Cramér distance. We show that the Cramér distance possesses all three desired properties, combining the best of the Wasserstein and Kullback-Leibler divergences. We give empirical results on a number of domains comparing these three divergences. To illustrate the practical relevance of the Cramér distance we design a new algorithm, the Cramér Generative Adversarial Network (GAN), and show that it has a number of desirable properties over the related Wasserstein GAN.

The manuscript proposes to use the Cramer distance as a measure between distributions (acting as a loss) when optimizing
an objective function using stochastic gradient descent (SGD). Cramer distance is a Bregman divergence and is a member of the Lp family of divergences.  Here a "distance" means a symmetric divergence measure that satisfies the relaxed triangle inequality. The motivation for using the Cramer distance is that it has unbiased sample gradients while still enjoying some other properties such as scale sensitivity and sum invariant. The authors also proof that for the Bernoulli distribution, there is a lower bound independent of the sample size for the deviation between the gradient of the Cramer distance, and the expectation of the estimated gradient of the Cramer distance. Then, the multivariate case of the Cramer distance, called the energy distance, is also briefly presented. The paper closes with some experiments on ordinal regression using neural networks and training GANs using the Cramer distance.
In general, the manuscript is well written and the ideas are smoothly presented. While the manuscript gives some interesting insights, I find that the contribution could have been explained in a more broader sense, with a stronger compelling message.
Some remarks and questions:
1.	The KL divergence considered here is sum invariant but not scale sensitive, and has unbiased sample gradients. The
	authors are considering here the standard (asymmetric) KL divergence (sec. 2.1). Is it the case that losing scale
	sensitivity make the KL divergence insensitive to the geometry of the outcomes? or is it due to the fact the KL
	divergence is not symmetric? or ?
2.	The main argument for the paper is that the simple sample-based estimate for the gradient using the Wasserstein
	metric is a biased estimate for the true gradient of the Wasserstein distance, and hence it is not favored with
	SGD-type algorithms. Are there any other estimators in the literature for the gradient of the Wasserstein distance?
	Was this issue overlooked in the literature?
3.	I am not sure if a biased estimate for the gradient will lead to a ``wrong minimum'' in an energy space that has
	infinitely many local minima.  Of course one should use an unbiased estimate for the gradient whenever this is possible.
	However, even when this is possible, there is no guarantee that this will consistently lead to deeper and ``better''
	minima, and there is no guarantee as well that these deep local minima reflect meaningful results.
4.	To what extent can one generalize theorem 1 to other probability distributions (continuous and discrete) and to the
	multivariate cases as well?
5.	I also don't think that the example given in sec. 4.2 and depicted in Fig. 1 is the best and simplest way to illustrate
	the benefit of Cramer distance over Wasserstein. Similarly, the experiments for the multivariate case using GANs and
	Neural Networks do not really deliver tangible, concrete and conclusive results. Partly, these results are very
       qualitative, which can be understood within the context of GANs. However, the authors could have used other
       models/algorithms where they can obtain concrete quantitative results (for this type of contribution). In addition,
	such sophisticated models (with various hyper-parameters) can mask the true benefit for the Cramer distance, and can
	also mask the extent of how good/poor is the sample estimate for the Wasserstein gradient.


Second-order methods for neural network optimization have several advantages over methods based on first-order gradient descent, including better scaling to large mini-batch sizes and fewer updates needed for convergence. But they are rarely applied to deep learning in practice because of high computational cost and the need for model-dependent algorithmic variations. We introduce a variant of the Hessian-free method that leverages a block-diagonal approximation of the generalized Gauss-Newton matrix. Our method computes the curvature approximation matrix only for pairs of parameters from the same layer or block of the neural network and performs conjugate gradient updates independently for each block. Experiments on deep autoencoders, deep convolutional networks, and multilayer LSTMs demonstrate better convergence and generalization compared to the original Hessian-free approach and the Adam method.

Summary:
The paper considers second-order optimization methods for training of neural networks.
In particular, the contribution of the paper is a Hessian-free method that works on blocks of parameters (this is a user defined splitting of the parameters in blocks, e.g., parameters of each layer is one block, or parameters in several layers could constitute a block).
This results into a block-diagonal approximation to the curvature matrix, in order to improve Hessian-free convergence properties: in the latter, a single step might require many CG steps, so the benefit from using second-order information is not apparent.
This is mainly an experimental work, where the authors show the merits of their approach on deep autoencoders, convolutional networks and LSTMs: results show favourable performance compared to the original Hessian-free approach and the Adam method.
Originality:
The paper is based on the works of Collobert (2004) and Le Roux et al. (2008), as well as the work of Martens: the twist is that each layer of the neural network is considered a parameter block, so that gradient interactions among weights in a single layer are more useful than those between weights in different layers. This increases the separability of the problem and reduces the complexity.
Importance:
Understanding the difference between first- and second-order methods for NN training is an important topic. Using second-order methods could be considered at its infancy, compared to the wide variety of first-order methods. Having new results on second-order methods with interesting results would definitely attract some attention at the conference.
Presentation/Clarity:
The paper is well structured and well written. The authors clearly place their work w.r.t. state of the art and previous works, so that it is clear what is new and what is known.
Comments:
1. It is not clear why the deficiency of first-order methods on training NNs with big batches motivates us to turn into second-order methods. Is there a reasoning for this statement? Or is it just because second-order methods are kind-of the only other alternative we have?
2. Assuming we can perform a second-order method, like Newton's method, on a deep NN. Since originally Newton's method was designed to find solutions that have gradient equal to zero, and since NNs have saddle points (probably many more than local minima), even if we could perfectly perform second-order Newton motions, there is no guarantee whether we converge to a local minimum or a saddle point. However, since we perform Newton's method approximately in practice, this might help escaping saddle points. Any comment on this aspect (I'm not aware whether this is already commented in Schraudolph 2002, where the Gauss-Newton matrix was proposed instead of the Hessian)?


Traditional recurrent neural network (RNN) or convolutional neural network (CNN) based sequence-to-sequence model can not handle tree structural data well. To alleviate this problem, in this paper, we propose a tree-to-tree model with specially designed encoder unit and decoder unit, which recursively encodes tree inputs into highly folded tree embeddings and decodes the embeddings into tree outputs. Our model could represent the complex information of a tree while also restore a tree from embeddings. We evaluate our model in random tree recovery task and neural machine translation task. Experiments show that our model outperforms the baseline model.

Summary: the paper proposes a tree2tree architecture for NLP tasks. Both the encoder and decoder of this architecture make use of memory cells: the encoder looks like a tree-lstm to encode a tree bottom-up, the decoder generates a tree top-down by predicting the number of children first. The objective function is a linear mixture of the cost of generating the tree structure and the target sentence. The proposed architecture outperforms recursive autoencoder on a self-to-self predicting trees, and outperforms an lstm seq2seq on En-Cn translation.
Comment:
- The idea of tree2tree has been around recently but it is difficult to make it work. I thus appreciate the authors’ effort. However, I wish the authors would have done it more properly.
- The computation of the encoder and decoder is not novel. I was wondering how the encoder differs from tree-lstm. The decoder predicts the number of children first, but the authors don’t explain why they do that, nor compare this to existing tree generators.
- I don’t understand the objective function (eq 4 and 5). Both Ls are not cross-entropy because label and childnum are not probabilities. I also don’t see why using Adam is more convenient than using SGD.
- I think eq 9 is incorrect, because the decoder is not Markovian. To see this we can look at recurrent neural networks for language modeling: generating the current word is conditioning on the whole history (not only the previous word).
- I expect the authors would explain more about how difficult the tasks are (eg. some statistics about the datasets), how to choose values for lambda, what the contribution of the new objective is.
About writing:
- the paper has so many problems with wording, e.g. articles, plurality.
- many terms are incorrect, e.g. “dependent parsing tree” (should be “dependency tree”), “consistency parsing” (should be “constituency parsing”)
- In 3.1, Socher et al. do not use lstm
- I suggest the authors to do some more literature review on tree generation


Published as a conference paper at ICLR 2018 COMPOSITIONAL OBVERTER COMMUNICATION LEARNING FROM RAW VISUAL INPUT
One of the distinguishing aspects of human language is its compositionality, which allows us to describe complex environments with limited vocabulary. Previously, it has been shown that neural network agents can learn to communicate in a highly structured, possibly compositional language based on disentangled input (e.g. handengineered features). Humans, however, do not learn to communicate based on well-summarized features. In this work, we train neural agents to simultaneously develop visual perception from raw image pixels, and learn to communicate with a sequence of discrete symbols. The agents play an image description game where the image contains factors such as colors and shapes. We train the agents using the obverter technique where an agent introspects to generate messages that maximize its own understanding. Through qualitative analysis, visualization and a zero-shot test, we show that the agents can develop, out of raw image pixels, a language with compositional properties, given a proper pressure from the environment.

This paper presents a technique for training a two-agent system to play a simple reference game involving recognition of synthetic images of a single object.  Each agent is represented by an RNN that consumes an image representation and sequence of tokens as input, and generates a binary decision as output. The two agents are initialized independently and randomly. In each round of training, one agent is selected to be the speaker and the other to be the listener. The speaker generates outputs by greedily selecting a sequence of tokens to maximize the probability of a correct recognition w/r/t the speaker's model. The listener then consumes these tokens, makes a classification decision, incurs a loss, and updates its parameters. Experiments find that after training, the two agents converge to approximately the same language, that this language contains some regularities, and that agents are able to successfully generalize to novel combinations of properties not observed during training.
While Table 3 is suggestive, this paper has many serious problems. There isn't an engineering contribution---despite the motivation at the beginning, there's no attempt to demonstrate that this technique could be used either to help comprehension of natural language or to improve over the numerous existing techniques for automatically learning communication protocols.  But this also isn't science: "generalization" is not the same thing as compositionality, and there's no testable hypothesis articulated about what it would mean for a language to be compositional---just the post-hoc analysis offered in Tables 2 & 3. I also have some concerns about the experiment in Section 3.3 and the overall positioning of the paper.
I want to emphasize that these results are cool, and something interesting might be going on here! But the paper is not ready to be published. I apologize in advance for the length of this review; I hope it provides some useful feedback about how future versions of this work might be made more rigorous.
WHAT IS COMPOSITIONALITY?
The title, introduction, and first three sections of this paper emphasize heavily the extent to which this work focuses on discovering "compositional" language. However, the paper doesn't even attempt to define what is meant by compositionality until the penultimate page, where it asserts that the ability to "accurately describe an object [...] not seen before" is "one of the marks of compositional language". Various qualitative claims are made that model outputs "seem to be" compositional or "have the strong flavor of" compositionality. Finally, the conclusion notes that "the exact definition of compositional language is somewhat debatable, and, to the best of our knowledge, there was no reliable way to check for the compositionality of an arbitrary language."
This is very bad.
It is true that there is not a universally agreed-upon definition of compositionality. In my experience, however, most people who study these issues do not (contra the citation-less 5th sentence of section 4) think it is simply an unstructured capacity for generalization.  And the implication that nobody else has ever attempted to provide a falsifiable criterion, or that this paper is exempt from itself articulating such a criterion, is totally unacceptable.  (You cannot put "giving the reader the tools to evaluate your current claims" in future work!)
If this paper wishes to make any claims about compositionality, it must _at a minimum_:
1. Describe a test for compositionality.
2. Describe in detail the relationship between the proposed test and other definitions of compositionality that exist in the literature.
3. If this compositionality is claimed to be "language-like", extend and evaluate the definition of compositionality to more complex concepts than conjunctions of two predicates.
Some thoughts to get you started: when talking about string-valued things, compositionality almost certainly needs to say something about _syntax_. Any definition you choose will be maximally convincing if it can predict _without running the model_ what strings will appear in the gray boxes in Figure 3.  Similarly if it can consistently generate analyses across multiple restarts of the training run.  The fact that analysis relies on seemingly arbitrary decisions to ignore certain tokens is a warning sign. The phenomenon where every color has 2--3 different names depending on the shape it's paired with would generally be called "non-compositional" if it appeared in a natural language.
This SEP article has a nice overview and bibliography: https://plato.stanford.edu/entries/compositionality/. But seriously, please, talk to a linguist.
MODEL
The fact that the interpretation model greedily chooses symbols until it reaches a certain confidence threshold would seem to strongly bias the model towards learning a specific communication strategy. At the same time, it's not actually possible to guarantee that there is a greedily-discoverable sequence that ever reaches the threshold! This fact doesn't seem to be addressed.
This approach also completely rules out normal natural language phenomena (consider "I know Mary" vs "I know Mary will be happy to see you"). It is at least worth discussing these limitations, and would be even more helpful to show results for other architectures (e.g. fixed-length codes or loss functions with an insertion penalty) as well.
There's some language in the appendix ("We noticed that a larger vocabulary and a longer message length helped the agents achieve a high communication accuracy more easily. But the resulting messages were challenging to analyze for compositional patterns.") that suggests that even the vague structure observed is hard to elicit, and that the high-level claim made in this paper is  less robust than the body suggests. It's _really_ not OK to bury this kind of information in the supplementary material, since it bears directly on your core claim that compositional structure does arise in practice. If the emergence of compositionality is sensitive to vocab size & message length, experiments demonstrating this sensitivity belong front-and-center in the paper.
EVALUATION
The obvious null hypothesis here is that unseen concepts are associated with an arbitrary (i.e. non-compositional) description, and that to succeed here it's enough to recognize this description as _different_ without understanding anything about its structure. So while this evaluation is obviously necessary, I don't think it's powerful enough to answer the question that you've posed. It would be helpful to provide some baselines for reference: if I understand correctly, guessing "0" identically gives 88% accuracy for the first two columns of Table 4, and guessing based on only one attribute gives 94%, which makes some of the numbers a little less impressive.
Perhaps more problematically, these experiments don't rule out the possibility that the model always guesses "1" for unseen objects. It would be most informative to hold out multiple attributes for each held-out color (& vice-versa), and evaluate only with speakers / listeners shown different objects from the held-out set.
POSITIONING AND MOTIVATION
The first sentence of this paper asserts that artificial general intelligence requires the ability to communicate with humans using natural language. This paper has nothing to do with AGI, humans, or human language; to be blunt, this kind of positioning is at best inappropriate and at worst irresponsible. It must be removed. For the assertion that "natural language processing has shown great progress", the paper provides a list of citations employing neural networks exclusively and beginning in 2014 (!). I would gently remind the authors that NLP research did not begin with deep learning, and that there might be slightly earlier evidence for their claim.
The attempt to cite relevant work in philosophy and psychology is commendable!  However, many of these citations are problematic, and some psycho-/historico-linguistic claims are still missing citations. A few examples: Ludwig Wittgenstein died in 1951, so it is somewhat surprising to see him cited for a 2010 publication (PI appeared in 1953); similarly Zipf (2016).  The application of this Zipf citation is dubious; the sentence preceded by footnote 7 is false and has nothing to do with the processes underlying homophony in natural languages. I would encourage you to consult with colleagues in the relevant fields.


We introduce an efficient way to increase the accuracy of convolution neural networks (CNNs) based on high model utilization without increasing any computational complexity. The proposed sparse-complementary convolution replaces regular convolution with sparse and complementary shapes of kernels, covering the same receptive field. By the nature of deep learning, high model utilization of a CNN can be achieved with more simpler kernels rather than fewer complex kernels. This simple but insightful model reuses of recent network architectures, ResNet and DenseNet, can provide better accuracy for most classification tasks (CIFAR-10/100 and ImageNet) compared to their baseline models. By simply replacing the convolution of a CNN with our sparse-complementary convolution, at the same FLOPs and parameters, we can improve top-1 accuracy on ImageNet by 0.33% and 0.18% for ResNet-101 and ResNet-152, respectively. A similar accuracy improvement could be gained by increasing the number of layers in those networks by ∼ 1.5×.

This paper introduces a new design of kernels in convolutional neural networks. The idea is to have sparse but complementary kernels with predefined patterns, which altogether cover the same receptive field as dense kernels. Because of the sparsity of such kernels, deeper or wider networks can be designed at the same computational cost as networks with dense kernels.
Strengths:
- The complementary kernels come at no loss compare to standard ones
- The resulting wider networks can achieve better accuracies than the original ones
Weaknesses:
- The proposed patterns are clear for 3x3 kernels, but no solution is proposed for other dimensions
- The improvement over the baseline is not very impressive
- There is no comparison against other strategies, such as 1xk and kx1 kernels (e.g., Ioannou et al. 2016)
Detailed comments:
- The separation into + and x patterns is quite clear for 3x3 kernels. However, two such patterns would not be sufficient for 5x5 or 7x7 kernels. This idea would have more impact if it generalized to arbitrary kernel dimensions.
- The improvement over the original models are of the order of less than 1 percent. I understand that such improvements are not easy to achieve, but one could wonder if they are not due to the randomness of initialization/mini-batches. It would be more meaningful to report average accuracies and standard deviations over several runs of each experiment.
- Section 4.4 briefly discusses the comparison with using 3x1 and 1x3 kernels, mentioning that an empirical comparison is beyond the scope of this paper. To me, this comparison is a must. In fact, the discussion in this section is not very clear to me, as it mentions additional experiments that I could not find (maybe I misunderstood the authors). What I would like to see is the results of a model based on the method of Ioannou et al, 2016 with the same number of FLOPS.
- In Section 2, the authors review ideas of so-called random kernel sparsity. Note that the work of Wen et al., 2016, and that of Alvarez & Salzmann, NIPS 2016, do not really impose random sparsity, but rather aim to cancel out entire kernels, thus reducing the size of the model and not requiring implementation overhead. They also do not require pre-training and re-training, but just a single training procedure. Note also that these methods often tend not to decrease accuracy, but rather even increase it (by a similar magnitude to that in this paper), for a more compact model.
- In the context of random sparsity, it would be worth citing the work of Collins & Kohli, 2014, Memory Bounded Deep Convolutional Networks.
- I am not entirely convinced by the discussion of the grouped sparsity method in Section 3.1. In fact, the order of the channels is arbitrary, since the kernels are learnt. Therefore, it seems to me that they could achieve the same result. Maybe the authors can clarify this?
- Is there a particular reason why the central points appears in both complementary kernels (+ and x)?
- Why did the authors change the training procedure of ResNets slightly compared to the original paper, i.e., 50k training images instead of 45k training + 5k validation? Did the baseline (original model) reported here also use 50k? What would the results be with 45k?
- Fig. 5 is not entirely clear to me. What was the width of each layer? The original one or the modified one?
- It would be interesting to report the accuracy of a standard ResNet with 1.325*width as a comparison, as well as the runtime of such a model.
- In Table 4, I find it surprising that there is an actual speedup for the model with larger width. I would have expected the same runtime. How do the authors explain this?


GAUSSIAN PROCESS BEHAVIOUR IN WIDE DEEP NEURAL NETWORKS
Whilst deep neural networks have shown great empirical success, there is still much work to be done to understand their theoretical properties. In this paper, we study the relationship between Gaussian processes with a recursive kernel definition and random wide fully connected feedforward networks with more than one hidden layer. We exhibit limiting procedures under which finite deep networks will converge in distribution to the corresponding Gaussian process. To evaluate convergence rates empirically, we use maximum mean discrepancy. We then exhibit situations where existing Bayesian deep networks are close to Gaussian processes in terms of the key quantities of interest. Any Gaussian process has a flat representation. Since this behaviour may be undesirable in certain situations we discuss ways in which it might be prevented.

In part 1, the authors introduce motivation for studying wide neural networks and summarize related work.
In part 2,  they present a theorem (main theoretical result) stating that under conditions on the weight priors, the output function of a multi-layer neural network (conditionally to a given input) weakly converges to a gaussian process as the size of the hidden layers go to infinity.
remark on theorem 1: This result generalizes a result proven in 2015 stating that the normality of a layer propagates to the next as the size of the first layer goes to infinity. The result stated in this paper is proven by bounding the gap between the output distribution and the corresponding gaussian process, and by propagating this bound across layers (appendix).
In part 3, the authors discuss the choice of a nonlinearity function that enables easy computation of the kernels introduced in the covariance matrix of the limit normal distribution. Their choice lands on ReLU.
In part 4, the focus is on the speed of the convergence presented in theorem 1. Experiments are conducted to show how the distance (maximum mean disrepancy) between the output distribution and its theoretical gaussian process limit vary when the sizes of the hidden layers increase. The results show that the convergence (in MMD) happens consistently, although it is slower when the number of hidden layers gets bigger.
In part 5, the authors compare the distributions (finite Bayesian deep networks and their analogues Gaussian processes) in yet another way: by studying their agreement in terms of inference. For this purpose, the authors chose several crieteria: the first two moments of the posterior, the log marginal likelihood and the predictive log-likelihood. The authors judge that the distributions agree on those criteria, but do not provide further analysis.
In part 6, now that It has been shown that the output distributions of Bayesian neural nets do not only weakly converge to Gaussian processes but also behave similarly in terms of inference, the authors discuss ways to avoid the gaussian process behaviour. Indeed, it seems that Gaussian processes with a fixed kernel cannot learn hierarchical representations, that are essential in deep learning.
The idea to avoid the Gaussian process behaviour is to contradict one of the hypothesis of the CLT (so that it does not hold anymore), either by controlling the size of intermediate layers, by using networks with infinite variance in the activities, or by choosing non-independent weights.
In part 7, it is concluded that the result that has been proven for size of layers going to infinity (Theorem 1) seems to empirically be verified on finite networks similar to those used in the literature. This can be used to simplify inference in cases were the gaussian process behaviour is desired, and opens questions on how to avoid this behaviour the rest of the time.
Pros: The authors line of thought of the authors is overall quite easy to follow. The main theoretical convergence result is stated early on, and the remaining of the article is dedicated to observing this result empirically from different angles (MMD, inference, predictive capability..). The last part contains a discussion concerning the extent to which it is actually a desired or a undesired result in classical deep learning use-cases, and the authors provide intuitive conditions under which the convergence would not hold. The stated theorem is a clear improvement on the past literature and is promising in a context where multi-layers neural networks are more and more studied.
Finally, the work is well documented.
Cons:
I have a some concerns with the main result (Theorem 1) and found that some of the notations / formulas were not very clear.
 Concerns with Theorem 1:
* at the end of the proof of Lemma 2, H_\mu is to be chosen large enough in order to get the \epsilon bound of the statement. However, I think that  H_\mu is constrained by the statement of Proposition 2, not to be larger than a constant times 2^(H_{\mu+1}). Isn't that a problem?
* In the proof of Lemma 4, it looks like matrix \Psi, from the schur decomposition of \tilde f, actually depends on H_{\mu-2}, thus making \psi_max depend on it too, as well as the final \beta bound, which would contradict the statement that it depends only on n and H_{\mu}. Could you please double check?
Unclear statements/notations:
* end of page 3, notations are not entirely consist with previous notations
* I do not understand which distribution is assumed on epsilon and gamma when taking the expectancy in equation (9).
* the notation x^(i) (in the theorem and the proof notably) could be changed, for the ^(i) index refers to the depth of the layer in the rest of the notations, and is here surprisingly referring to a set of observations.
* the statement of Theorem 1:
    * I would change "for a countable input set" to "for any countable input set", if this holds true.
    * does not say that the width has to go to infinity for the convergence to happen, which goes a bit in contradiction with the adjective "wide". However, the authors say that in practice, they use the identity as width function.
* I understood that the conclusion of part 3 was that the expectation of eq (9) was elegantly computable for certain non-linearity (including ReLU). However I don't see the link with the "recursive kernel" idea (maybe it's just the way to do the computation described in Cho&Saul(2009) ?)
Some places where it appears that there are minor mistakes:
* 7th line from the bottom of page 3, the vector f^{(2)}(x) contains f_i^{(1)}(x) but should contain f_i^{(2)}(x)
* last display of page 3: change x and x', and indicate upper limit of the sum
* please double check variances C_w and/or \hat{C}_w appearing in equations in (9) and (13).
* line 2 of second paragraph after equations (8) and (9). The authors refer to equation (8) concerning the independence of the components of the output. I think they rather wanted to refer to (9). Same for first sentence before eq (14).
* middle of page 12: matrix LY should be RY.


Deep learning algorithms achieve high classification accuracy at the expense of significant computation cost. To address this cost, a number of quantization schemes have been proposed -but most of these techniques focused on quantizing weights, which are relatively smaller in size compared to activations. This paper proposes a novel quantization scheme for activations during training -that enables neural networks to work well with ultra low precision weights and activations without any significant accuracy degradation. This technique, PArameterized Clipping acTivation (PACT), uses an activation clipping parameter α that is optimized during training to find the right quantization scale. PACT allows quantizing activations to arbitrary bit precisions, while achieving much better accuracy relative to published state-of-the-art quantization schemes. We show, for the first time, that both weights and activations can be quantized to 4-bits of precision while still achieving accuracy comparable to full precision networks across a range of popular models and datasets. We also show that exploiting these reduced-precision computational units in hardware can enable a super-linear improvement in inferencing performance due to a significant reduction in the area of accelerator compute engines coupled with the ability to retain the quantized model and activation data in on-chip memories.

The authors have addressed my concerns, and clarified a misunderstanding of the baseline that I had, which I appreciate. I do think that it is a solid contribution with thorough experiments. I still keep my original rating of the paper because the method presented is heavily based on previous works, which limits the novelty of the paper. It uses previously proposed clipping activation function for quantization of neural networks, adding a learnable parameter to this function.
_______________
ORIGINAL REVIEW:
This paper proposes to use a clipping activation function as a replacement of ReLu to train a neural network with quantized weights and activations. It shows empirically that even though the clipping activation function obtains a larger training error for full-precision model, it maintains the same error when applying quantization, whereas training with quantized ReLu activation function does not work in practice because it is unbounded.
The experiments are thorough, and report results on many datasets, showing that PACT can reduce down to 4 bits of quantization of weights and activation with a slight loss in accuracy compared to the full-precision model.
Related to that, it seams a bit an over claim to state that the accuracy decrease of quantizing the DNN with PACT in comparison with previous quantization methods is much less because the decrease is smaller or equal than 1%, when competing methods accuracy decrease compared to the full-precision model is more than 1%. Also, it is unfair to compare to the full-precision model using clipping, because ReLu activation function in full-precision is the standard and gives much better results, and this should be the reference accuracy. Also, previous methods take as reference the model with ReLu activation function, so it could be that in absolute value the accuracy performance of competing methods is actually higher than when using PACT for quantizing DNN.
OTHER COMMENTS:
- the list of contributions is a bit strange. It seams that the true contribution is number 1 on the list, which is to introduce the parameter \alpha in the activation function that is learned with back-propagation, which reduces the quantization error with respect to using ReLu as activation function. To provide an analysis of why it works and quantitative results, is part of the same contribution I would say.


Imitation learning relies on expert demonstrations. Existing approaches often require that the complete demonstration data, including sequences of actions and states are available. In this paper, we consider a realistic and more difficult scenario where a reinforcement learning agent only has access to the state sequences of an expert, while the expert actions are not available. Inferring the unseen expert actions in a stochastic environment is challenging and usually infeasible when combined with a large state space. We propose a novel policy learning method which only utilizes the expert state sequences without inferring the unseen actions. Specifically, our agent first learns to extract useful sub-goal information from the state sequences of the expert and then utilizes the extracted sub-goal information to factorize the action value estimate over state-action pairs and subgoals. The extracted sub-goals are also used to synthesize guidance rewards in the policy learning. We evaluate our agent on five Doom tasks. Our empirical results show that the proposed method significantly outperforms the conventional DQN method.

SIGNIFICANCE AND ORIGINALITY:
The authors propose to accelerate the learning of complex tasks by exploiting traces of experts.
Unlike the most common form of imitation learning or behavioral cloning, the authors
formulate their solution in the case where the expert’s state trajectory is observable,
but the expert’s actions are not. This is an important and useful problem in robotics and other
applications. Within this specific setting the authors differentiate their approach from others
by developing a solution that does NOT estimate an explicit dynamics model ( e.g.,  P( S’ | S, A ) ).
The benefits of not estimating an explicit action model are not really demonstrated in a clear way.
The author’s articulate a specific solution that provides heuristic guidance rewards that cause the
learner to favor actions that achieve subgoals calculated from expert behavior
and refactors the representation of the Q function so that it
has a component that is a function of the subgoal extracted from the expert.
These subgoals are linear functions of the expert’s change in state (or change in state features).
The resultant policy is a function of the expert traces on which it depends.
The authors show they can retrain a new policy that does not require the expert traces.
As far as I am aware, this is a novel approach to the problem.
The authors claim that this factorization is important and useful but the paper doesn’t
really illustrate this well.
They demonstrate the usefulness of the algorithm against a DQN baseline on Doom game problems.
The algorithm learns faster than unassisted DQN as shown by learning curve plots.
They also evaluate the algorithms on the quality of the final policies for their approach, DQN,
and  a supervised learning from demonstration approach ( LfD ) that requires expert actions.
The proposed approach does as well or better than competing approaches.
QUALITY
Ablation studies show that the guidance rewards are important to achieving the improved performance of the proposed method which is important confirmation that the architecture is working in the intended way. However, it would also be useful to do an ablation study of the “factorization” of action values.  Is this important to achieving better results as well or is the guidance reward enough? This seems like a key claim to establish.
CLARITY
The details of the memory based kernel density estimation and neural gradient training seemed
complicated by the way that the process was implemented. Is it possible to communicate
the intuitions behind what is going on?

I was able to work out the intuitions behind the heuristic rewards, but I still don’t clearly get
what the Q-value factorization is providing:
To keep my text readable, I assume we are working in feature space
instead of state space and use different letters for learner and expert:
   Learner: S = \phi(s)
   Expert’s i^th state visit:  Ei = \phi( \hat{s}_i }  where Ei’ is the successor state to Ei
The paper builds upon approximate n-step discrete-action Q-learning
where the Q value for an action is a linear function of the state features:
    Qp(S,a) = Wa S + Ba
where parameters p = ( Wa, Ba ).
After observing an experience ( S,A,R,S’ ) we use Bellman Error as a loss function to optimize Qp for parameter p.
I ignore the complexities of n-step learning and discount factors for clarity.
    Loss = E[    R + MAXa’ Qp(S’,a’)    -   Qp(S,a)   ]
The authors suggest we can augment the environment reward R
with a heuristic reward Rh proportional to the similarity between
the learner “subgoal" and the expert “subgoal" in similar states.
The authors propose to use cosine distance between representations
of what they call the “subgoals” of learner and expert.
A subgoal is defined as a linear transformation of the distance traveled by an agent during a transition.
The heuristic reward is proportional to the cosine distance between the learner and expert “subgoals"
   Rh = B  <   Wv LearnerDirectionInStateS,
                     Wv ExpectedExpertDirectionInStatesSimilarToS   >
The learner’s direction in state S is just (S-S’) in feature space.
The authors model the behavior of the expert as a kernel density type approximator
giving the expected direction of the expert starting from a states similar to the one the learner is in.
Let < Wk S, Wk Ej > be a weighted similarity between learner state features S and expert state features Ej
and Ej’ be the successor state features encountered by the expert.
Then the expected expert direction for learner state S is:
     SUMj  < Wk S, Wk Ej > ( Ej - Ej’ )
Presumably the linear Wk transform helps us pick out the important dimensions of similarity between S and Ej.
Mapping the learner and expert directions into subgoal space using Wv, the heuristic reward is
   Rh = B <   Wv (S-S’),
                    Wv SUMj  < Wk S, Wk Ej > ( Ej - Ej’ ) >
I ignore the ReLU here, but I assume that is operates element-wise and just clips negative values?
There is only one layer here so we don’t have complex non-linear things going on?
In addition to introducing a heuristic reward term, the authors propose to alter the Q-function
to be specific to the subgoal.
   Q( s,a,g ) = g(S) Wa S + Ba
The subgoal is the same as the first part, namely a linear transform of the expected expert direction in
states similar to state S.
    g(S) =  Wv   SUMj  < Wk S, Wk Ej >  ( Ej - Ej’ )
So in some sense, the Q function is really just a function of S, as g is calculated from S.
    Q( S,a ) = g(S) Wa S + Ba
So this allows the Q-function more flexibility to capture each subgoal in a different linear space?
I don’t really get the intuition behind this formulation. It allows the subgoal to adjust the value
of the underlying model? Essentially the expert defines a new Q-value problem at every state
for the learner? In some sense are we are defining a model for the action taken by the expert?
ADDITIONAL THOUGHTS
While the authors compare to an unassisted baseline, they don’t compare to methods that use an action model
which is not a fatal flaw but would have been nice.
One can imagine there might be scenarios where the local guidance rewards of this
form could be problematic, particularly in scenarios where the expert and learner are not identical
and it is possible to return to previous states, such as the grid worlds the authors discuss:
If the expert’s first few transitions were easily approximable,
the learner would get local rewards that cause it to mimic expert behavior.
However, if the next step in the expert’s path was difficult to approximate,
then the reward for imitating the expert would be lower.
Would the learner then just prefer to go back towards those states that it can approximate and endlessly loop?
In this case, perhaps expressing heuristic rewards as potentials as described in Ng’s shaping paper might solve the problem.
PROS AND CONS
Important problem generally. Avoiding the estimation of a dynamics model was stated as a given, but perhaps more could be put into motivating this goal. Hopefully it is possible to streamline the methodology section to communicate the intuitions more easily.


Published as a conference paper at ICLR 2018 QUANTITATIVELY EVALUATING GANS WITH DIVERGENCES PROPOSED FOR TRAINING
Generative adversarial networks (GANs) have been extremely effective in approximating complex distributions of high-dimensional, input data samples, and substantial progress has been made in understanding and improving GAN performance in terms of both theory and application. However, we currently lack quantitative methods for model assessment. Because ot this, while many GAN variants being proposed, we have relatively little understanding of their relative abilities. In this paper, we evaluate the performance of various types of GANs using divergence and distance functions typically used only for training. We observe consistency across the various proposed metrics and, interestingly, the test-time metrics do not favour networks that use the same training-time criterion. We also compare the proposed metrics to human perceptual scores.

This paper proposes using divergence and distance functions typically used for generative model training to evaluate the performance of various types of GANs. Through numerical evaluation, the authors observed that the behavior is consistent across various proposed metrics and the test-time metrics do not favor networks that use the same training-time criterion.
More specifically, the evaluation metric used in the paper are: 1) Jensen-Shannon divergence, 2) Constrained Pearson chi-squared, 3) Maximum Mean Discrepancy, 4) Wasserstein Distance, and 5) Inception Score. They applied those metrics to compare three different GANs: the standard DCGAN, Wasserstein DCGAN, and LS-DCGAN on MNIST and CIFAR-10 datasets.
Summary:
——
In summary, it is an interesting topic, but I think that the paper does not have sufficient novelty. Some empirical results are still preliminary. It is hard to judge the effectiveness of the proposed metrics for model selection and is not clear that those metrics are better qualitative descriptors to replace visual assessment. In addition, the writing should be improved. See comments below for details and other points.
Comments:
——
1.	In Section 3, the evaluation metrics are existing metrics and some of them have already been used in comparing GAN models.  Maximum mean discrepancy has been used before in work by Yujia Li et al. (2016, 2017)
2.	In the experiments, the proposed metrics were only tested on small scale datasets; the authors should evaluate on larger datasets such as CIFAR-100, Toronto Faces, LSUN bedrooms or CelebA.
3.	In the experiments, the authors noted that “Gaussian observable model might not be the ideal assumption for GANs. Moreover, we observe a high log-likelihood at the beginning of training, followed by a drop in likelihood, which then returns to the high value, and we are unable to explain why this happens.” Could the authors give explanation to this phenomenon? The authors should look into this more carefully.
4.	In algorithm 1, it seems that the distance is computed via gradient decent. Is it possible to show that the optimization always converges? Is it meaningful to compare the metrics if some of them cannot be properly computed?
5.     With many different metrics for assessing GANs, how should people choose? How do we trust the scores? Recently, Fréchet Inception Distance (FID) was proposed to evaluate the samples generated from GANs (Heusel et al. 2017), how are the above scores compared with FID?
Minor Comments:
——
1.	Writing should be fixed: “It seems that the common failure case of MMD is when the mean pixel intensities are a better match than texture matches (see Figure 5), and the common failure cases of IS happens to be when the samples are recognizable textures, but the intensity of the samples are either brighter or darker (see Figure 2).”


Activity of populations of sensory neurons carries stimulus information in both the temporal and the spatial dimensions. This poses the question of how to compactly represent all the information that the population codes carry across all these dimensions. Here, we developed an analytical method to factorize a large number of retinal ganglion cells' spike trains into a robust low-dimensional representation that captures efficiently both their spatial and temporal information. In particular, we extended previously used single-trial space-by-time tensor decomposition based on non-negative matrix factorization to efficiently discount pre-stimulus baseline activity. On data recorded from retinal ganglion cells with strong pre-stimulus baseline, we showed that in situations where the stimulus elicits a strong change in firing rate, our extensions yield a boost in stimulus decoding performance. Our results thus suggest that taking into account the baseline can be important for finding a compact information-rich representation of neural activity.

This study proposes the use of non-negative matrix factorization accounting for baseline by subtracting the pre-stimulus baseline from each trial and subsequently decompose the data using a 3-way factorization thereby identifying spatial and temporal modules as well as their signed activation. The method is used on data recorded from mouse and pig retinal ganglion cells of time binned spike trains providing improved performance over non-baseline corrected data.
Pros:
The paper is well written, the analysis interesting and the application of the Tucker2 framework sound. Removing baseline is a reasonable step and the paper includes analysis of several spike-train datasets. The analysis of the approaches in terms of their ability to decode is also sound and interesting.
Cons:
I find the novelty of the paper limited:
The authors extend the work by (Onken et al. 2016) to subtract baseline (a rather marginal innovation) of this approach. To use a semi-NMF type of update rule (as proposed by Ding et al .2010) and apply the approach to new spike-train datasets evaluating performance by their decoding ability (decoding also considered in Onken et al. 2016).
Multiplicative update-rules are known to suffer from slow-convergence and I would suspect this also to be an issue for the semi-NMF update rules. It would therefore be relevant and quite easy to consider other approaches such as active set or column wise updating also denoted HALS which admit negative values in the optimization, see also the review by N. Giles
https://arxiv.org/abs/1401.5226
as well as for instance:
Nielsen, Søren Føns Vind, and Morten Mørup. "Non-negative tensor factorization with missing data for the modeling of gene expressions in the human brain." Machine Learning for Signal Processing (MLSP), 2014 IEEE International Workshop on. IEEE, 2014.
It would improve the paper to also discuss that the non-negativity constrained Tucker2 model may be subject to local minima solutions and have issues of non-uniqueness (i.e. rotational ambiguity). At least local minima issues could be assessed using multiple random initializations.
The results are in general only marginally improved by the baseline corrected non-negativity constrained approach. For comparison the existing methods ICA, Tucker2 should also be evaluated for the baseline corrected data, to see if it is the constrained representation or the preprocessing influencing the performance. Finally, how performance is influenced by dimensionality P and L should also be clarified.
It seems that it would be naturally to model the baseline by including mean values in the model rather than treating the baseline as a preprocessing step. This would bridge the entire framework as one model and make it potentially possible to avoid structure well represented by the Tucker2 representation to be removed by the preprocessing.
Minor:
The approach corresponds to a Tucker2 decomposition with non-negativity constrained factor matrices and unconstrained core - please clarify this as you also compare to Tucker2 in the paper with orthogonal factor matrices.
Ding et al. in their semi-NMF work provide elaborate derivation with convergence guarantees.  In the present paper these details are omitted and it is unclear how the update rules are derived from the KKT conditions and the Lagrange multiplier and how they differ from standard semi-NMF, this should be better clarified.


Published as a conference paper at ICLR 2018 TEMPORALLY EFFICIENT DEEP LEARNING WITH SPIKES
The vast majority of natural sensory data is temporally redundant. For instance, video frames or audio samples which are sampled at nearby points in time tend to have similar values. Typically, deep learning algorithms take no advantage of this redundancy to reduce computations. This can be an obscene waste of energy. We present a variant on backpropagation for neural networks in which computation scales with the rate of change of the data -not the rate at which we process the data. We do this by implementing a form of Predictive Coding wherein neurons communicate a combination of their state, and their temporal change in state, and quantize this signal using Sigma-Delta modulation. Intriguingly, this simple communication rule give rise to units that resemble biologically-inspired leaky integrate-and-fire neurons, and to a spike-timing-dependent weight-update similar to Spike-Timing Dependent Plasticity (STDP), a synaptic learning rule observed in the brain. We demonstrate that on MNIST, on a temporal variant of MNIST, and on Youtube-BB, a dataset with videos in the wild, our algorithm performs about as well as a standard deep network trained with backpropagation, despite only communicating discrete values between layers.

This paper presents a novel method for spike based learning that aims at reducing the needed computation during learning and testing when classifying temporal redundant data. This approach extends the method presented on Arxiv on Sigma delta quantized networks (Peter O’Connor and Max Welling. Sigma delta quantized networks. arXiv preprint arXiv:1611.02024, 2016b.). Overall, the paper is interesting and promising; only a few works tackle the problem of learning with spikes showing the potential advantages of such form of computing. The paper, however, is not flawless. The authors demonstrate the method on just two datasets, and effectively they show results of training only for Feed-Forward Neural Nets (the authors claim that “the entire spiking network end-to-end works” referring to their pre-trained VGG19, but this paper presents only training for the three top layers). Furthermore, even if suitable datasets are not available, the authors could have chosen to train different architectures. The first dataset is the well-known benchmark MNIST also presented in a customized Temporal-MNIST. Although it is a common base-line, some choices are not clear: why using a FFNN instead that a CNN which performs better on this dataset; how data is presented in terms of temporal series – this applies to the Temporal MNIST too; why performances for Temporal MNIST – which should be a more suitable dataset — are worse than for the standard MNIST; what is the meaning of the right column of Figure 5 since it’s just a linear combination of the GOps results. For the second dataset, some points are not clear too: why the labels and the pictures seem not to match (in appendix E); why there are more training iterations with spikes w.r.t. the not-spiking case. Overall, the paper is mathematically sound, except for the “future updates” meaning which probably deserves a clearer explanation. Moreover, I don’t see why the learning rule equations (14-15) are described in the appendix, while they are referred constantly in the main text. The final impression is that the problem of the dynamical range of the hidden layer activations is not fully resolved by the empirical solution described in Appendix D: perhaps this problem affects CCNs more than FFN.
Finally, there are some minor issues here and there (the authors show quite some lack of attention for just 7 pages):
-	Two times “get” in “we get get a decoding scheme” in the introduction;
-	Two times “update” in “our true update update as” in Sec. 2.6;
-	Pag3 correct the capital S in 2.3.1
-	Pag4 Figure 1 increase font size (also for Figure2); close bracket after Equation 3; N (number of spikes) is not defined
-	Pag5 “one-hot” or “onehot”;
-	in the inline equation the sum goes from n=1 to S, while in eq.(8) it goes from n=1 to N;
-	Eq(10)(11)(12) and some lines have a typo (a \cdot) just before some of the ws;
-	Pag6 k_{beta} is not defined in the main text;
-	Pag7 there are two “so that” in 3.1; capital letter “It used 32x10^12..”; beside, here, why do not report the difference in computation w.r.t. not-spiking nets?
-	Pag7 in 3.2 “discussed in 1” is section 1?
-	Pag14 Appendix E, why the labels don’t match the pictures;
-	Pag14 Appendix F, explain better the architecture used for this experiment.


The increasing demand for neural networks (NNs) being employed on embedded devices has led to plenty of research investigating methods for training low precision NNs. While most methods involve a quantization step, we propose a principled Bayesian approach where we first infer a distribution over a discrete weight space from which we subsequently derive hardware-friendly low precision NNs. To this end, we introduce a probabilistic forward pass to approximate the intractable variational objective that allows us to optimize over discrete-valued weight distributions for NNs with sign activation functions. In our experiments, we show that our model achieves state of the art performance on several real world data sets. In addition, the resulting models exhibit a substantial amount of sparsity that can be utilized to further reduce the computational costs for inference.

In this work, discrete-weight NNs are trained using the variational Bayesian framework, achieving similar results to other state-of-the-art models. Weights use 3 bits on the first layer and are ternary on the remaining layers.
- Pros:
The paper is well-written and connections with the literature properly established.
The approach to training discrete-weights NNs, which is variational inference, is more principled than previous works (but see below).
- Cons:
The authors depart from the original motivation when the central limit theorem is invoked. Once we approximate the activations with Gaussians, do we have any guarantee that the new approximate lower bound is actually a lower bound? This is not discussed. If it is not a lower bound, what is the rationale behind maximizing it? This seems to place this work very close to previous works, and not in the "more principled" regime the authors claim to seek.
The likelihood weighting seems hacky. The authors claim "there are usually many more NN weights than there are data samples". If that is the case, then it seems that the prior dominating is indeed the desired outcome. A different, more flat prior (or parameter sharing), can be used, but the described reweighting seems to be actually breaking a good property of Bayesian inference, which is defecting to the prior when evidence is lacking.
In terms of performance (Table 1), the proposed method seems to be on par with existing ones. It is unclear then what the advantage of this proposal is.
Sparsity figures are provided for the current approach, but those are not contrasted with existing approaches. Speedup is claimed with respect to an NN with real weights, but not with respect existing NNs with binary weights, which is the appropriate baseline.
- Minor comments:
Page 3: Subscript t and variable t is used for the targets, but I can't find where it is defined.
Only the names of the datasets used in the experiments are given, but they are not described, or even better, shown in pictures (maybe in a supplementary).
The title of the paper says "discrete-valued NNs". The weights are discrete, but the activations and outputs are continuous, so I find it confusing. As a contrast, I would be less surprised to hear a sigmoid belief network called a "discrete-valued NN", even though its weights are continuous.


Published as a conference paper at ICLR 2018 CERTIFIED DEFENSES AGAINST ADVERSARIAL EX- AMPLES
While neural networks have achieved high accuracy on standard image classification benchmarks, their accuracy drops to nearly zero in the presence of small adversarial perturbations to test inputs. Defenses based on regularization and adversarial training have been proposed, but often followed by new, stronger attacks that defeat these defenses. Can we somehow end this arms race? In this work, we study this problem for neural networks with one hidden layer. We first propose a method based on a semidefinite relaxation that outputs a certificate that for a given network and test input, no attack can force the error to exceed a certain value. Second, as this certificate is differentiable, we jointly optimize it with the network parameters, providing an adaptive regularizer that encourages robustness against all attacks. On MNIST, our approach produces a network and a certificate that no attack that perturbs each pixel by at most = 0.1 can cause more than 35% test error.

This paper develops a new differentiable upper bound on the performance of classifier when the adversarial input in l_infinity is assumed to be applied.
While the attack model is quite general, the current bound is only valid for linear and NN with one hidden layer model, so the result is quite restrictive.
However the new bound is an "upper" bound of the worst-case performance which is very different from the conventional sampling based "lower" bounds. Therefore minimizing this upper bound together with a classification loss makes perfect sense and provides a theoretically sound approach to train a robust classifier.
This paper provides a gradient of this new upper bound with respect to model parameters so we can apply the usual first order optimization scheme to this joint optimization (loss + upper bound).
In conclusion, I recommend this paper to be accepted, since it presents a new and feasible direction of a principled approach to train a robust classifier, and the paper is clearly written and easy to follow.

There are possible future directions to be developed.
1. Apply the sum-of-squares (SOS) method.
The paper's SDP relaxation is the straightforward relaxation of Quadratic Program (QP), and in terms of SOS relaxation hierarchy, it is the first hierarchy. One can increase the complexity going beyond the first hierarchy, and this should provides a computationally more challenging but tighter upper bound.
The paper already mentions about this direction and it would be interesting to see the experimental results.
2. Develop a similar relaxation for deep neural networks.
The author already mentioned that they are pursuing this direction. While developing the result to the general deep neural networks might be hard, residual networks maybe fine thanks to its structure.


Auto-encoders are commonly used for unsupervised representation learning and for pre-training deeper neural networks. When its activation function is linear and the encoding dimension (width of hidden layer) is smaller than the input dimension, it is well known that auto-encoder is optimized to learn the principal components of the data distribution Oja (1982). However, when the activation is nonlinear and when the width is larger than the input dimension (overcomplete), auto-encoder behaves differently from PCA, and in fact is known to perform well empirically for sparse coding problems. We provide a theoretical explanation for this empirically observed phenomenon, when rectified-linear unit (ReLu) is adopted as the activation function and the hidden-layer width is set to be large. In this case, we show that, with significant probability, initializing the weight matrix of an auto-encoder by sampling from a spherical Gaussian distribution followed by stochastic gradient descent (SGD) training converges towards the ground-truth representation for a class of sparse dictionary learning models. In addition, we can show that, conditioning on convergence, the expected convergence rate is O( 1 t ), where t is the number of updates. Our analysis quantifies how increasing hidden layer width helps the training performance when random initialization is used, and how the norm of network weights influence the speed of SGD convergence.

This paper shows that an idealized version of stochastic gradient descent converges when learning autoencoders with ReLu non-linearities under strong sparsity assumptions. Convergence rates are also determined. The result is another one in the emerging line of proving convergence guarantees for non-convex optimization problems arising in machine learning, and aims to explain certain phenomena experienced in practice.
The paper is generally nicely written, providing intuitions, but there are several typos (both in the text and in the math, e.g., missing indices), which should also be corrected.
On the negative side, while the proof technique in general looks plausible, there seem to be some mistakes in the derivations, which must be corrected before the paper can be accepted. Also, the assumptions in the in the paper seem quite restrictive, and their implications are not discussed thoroughly.
The assumptions are the following:
1. The input data is coming from a mixture distribution, in the form x=w_I + eps, where {w_1,...,w_k} is a collection of unit vectors, I is uniform in {1,...,K}, eps is some noise (independent for each sample).
2. The maximum norm of the noise is O(1/k).
3. The number n of hidden neurons in the autoencoder is Omega(k) (this is not explicitly assumed but is necessary to make the probability of "incorrect" initialization small as well as the results to hold).
Under these assumptions it is shown that the weights of the autoencoder converge to the centers {w_1,...,w_k} (i.e., for any i the autoencoder has at least one weight converging to w_i). The rate of convergence depends on the coherence of the vectors w_i: the less coherent they are the faster the convergence is.
First notice that some assumptions are missing from the main statement, as the error probability delta is certainly connected to the probability of incorrect initialization: when n=1<k, the convergence result clearly cannot hold. This comes from the mistake that in Theorem 3 you state the bound for the probability P(F^\infty) instead of the conditional probability P(F^\infty|E_o) (this is present everywhere in the proof). Theorem 3 should also depend on delta_o, which is used in the definition of F^\infty.
Theorem 2 also seems incorrect. Intuitively, the question is why it cannot happen that two neurons contribute to reproducing a given w_i, and so neither of their weights converge to w_i: E.g., assuming that {w_1,...,w_k,w_1',...,w_k'} form an orthogonal system and the noise is 0, the weight matrix of size n=2k defined as W_{2i-1,*}^T = 1/sqrt{2}(w_i + w'_i) and W_{2i,*}^T=1/sqrt{2}(w_i - w'_i), i \in [k], with 0 bias can exactly recover any x=w_i (indeed, W_{2j-1,*} x= W_{2j,*} x = 1/sqrt{2}, while the other products are 0, and so W^T W x = W^T W w_j = 1/sqrt{2}(W_{2j-1,*}+W_{2j,*})^T = w_j). Then SGD does not change the weights and hence cannot recover the original weights {w_i }, in particular, it cannot increase the coherence in any step, contradicting Theorem 2. This counterexample can be extended even to the situation when k>d, as--in fact--we only need that the existence of a single j such that w_j and w'_j are orthogonal and also orthogonal to the other basis vectors.
The assumptions are also very strange in the sense that the norm of the noise is bounded by O(1/k), thus the more modes the input distribution has the more separable they become. What motivates this scaling? Furthermore, the parameters of the algorithm for which the convergence is claimed heavily depend on the problem parameters, which are not known. How can you instantiate the algorithm then (accepting the ideal definition of b)? What are the consequences?
Given the above, at this point I cannot recommend the paper for acceptance. However, if the above problems are resolved, I would be very happy to see the paper at the conference.
Other comments
-----------------------
- Add a short derivation why the weights of the autoencoder should converge to the w_i.
- Definition 3: C_j is not defined in the main text.
- While it is mentioned multiple times that the interesting regime is d<n, this is actually never used, nor needed (personally, I have never seen such an autoencoder--please give some references). What is really needed is n>k, which is natural if one wants to preserve the information, and also k>d for a rich family of distributions.
- The area of the spherical cap is well understood (up to multiplicative constants), and better bounds than yours are readily available: with a cap of height 1-t, for sqrt{2/d}<t<1, the relative surface of the cap is between P/6 and P/2 where
P=1/(t \sqrt{d}) (1-t^2)^{(d-1)/2}; see, e.g., A. Brieden, P. Gritzmann, R. Kannan, V. Klee, L. Lovasz, and M. Simonovits. Deterministic and randomized polynomial-time approximation of radii. Mathematika. A Journal of Pure and Applied Mathematics, 48(1-2):63–105, 2001.
- The notation section should be brought forward (or referred the fist time the notation is actually used).
- Instead of unit spherical Gaussian you could simply say uniform distribution on the unit sphere
- While Algorithm 1 is called "norm-controlled SGD training," it does not control the norm at all.


Auto-encoders compress input data into a latent-space representation and reconstruct the original data from the representation. This latent representation is not easily interpreted by humans. In this paper, we propose training an auto-encoder that encodes input text into human-readable sentences. The auto-encoder is composed of a generator and a reconstructor. The generator encodes the input text into a shorter word sequence, and the reconstructor recovers the generator input from the generator output. To make the generator output human-readable, a discriminator restricts the output of the generator to resemble human-written sentences. By taking the generator output as the summary of the input text, abstractive summarization is achieved without document-summary pairs as training data. Promising results are shown on both English and Chinese corpora.

Summary: In this work, the authors propose a text reconstructing auto encoder which takes a sentence as the input sequence and an integrated text generator generates another version of the input text while a reconstructor determines how well this generated text reconstructs the original input sequence. The input to the discriminator (as real data) is a sentence that summarizes the ground truth sentences (rather than the ground truth sentences themselves). The experiments are conducted in two datasets of English and Chinese corpora.
Strengths:
The proposed idea of generating text using summary sentences is new.
The model overview in Figure 1 is informative.
The experiments are conducted on English and Chinese corpora, comparison with competitive baselines are provided.
Weaknesses:
The paper is poorly written which makes it difficult to understand. The second paragraph in the introduction is quite cryptic. Even after reading the entire paper a couple of times, it is not clear how the summary text is obtained, e.g. do the authors ask annotators to read sentences and summarize them? If so, based on which criteria do the annotators summarize text, how many annotators are there? Similarly, if so this would mean that the authors use additional supervision than the compared models. Please clarify how the summary text is obtained.
In footnote 1, the authors mention “seq2seq2seq2” term which they do not explain anywhere in the text.
No experiments that generate raw text (without using summaries) are provided. It would be interesting to see if GAN learns to memorize the ground truth sentences or generates sentences with enough variation.
In the English Gigaword dataset the results consistently drop compared to WGAN. This behavior is observed for both the unsupervised setting and two versions of transfer learning settings. There are too few qualitative results: One positive qualitative result is provided in Figure 3 and one negative qualitative result is provided in Figure 4. Therefore, it is not easy for the reader to judge the behavior of the model well.
The choice of the evaluation metric is not well motivated. The standard measures in the literature also include METEOR, CIDER and SPICE. It would be interesting to see how the proposed model performs in these additional criteria. Moreover, the results are not sufficiently discussed.
As a general remark, although the idea presented in this paper is interesting, both in terms of writing and evaluation, this paper has not yet reached the maturity expected from an ICLR paper. Regarding writing, the definite and indefinite articles are sometimes missing and sometimes overused, similarly most of the times there is a singular/plural mismatch. This makes the paper very difficult to read. Often the reader needs to guess what is actually meant. Regarding the experiments, presenting results with multiple evaluation criteria and showing more qualitative results would improve the exposition.
Minor comments:
Page 5: real or false —> real or fake (true or false)
	     the lower loss it get —> ?


To provide principled ways of designing proper Deep Neural Network (DNN) models, it is essential to understand the loss surface of DNNs under realistic assumptions. We introduce interesting aspects for understanding the local minima and overall structure of the loss surface. The parameter domain of the loss surface can be decomposed into regions in which activation values (zero or one for rectified linear units) are consistent. We found that, in each region, the loss surface have properties similar to that of linear neural networks where every local minimum is a global minimum. This means that every differentiable local minimum is the global minimum of the corresponding region. We prove that for a neural network with one hidden layer using rectified linear units under realistic assumptions. There are poor regions that lead to poor local minima, and we explain why such regions exist even in the overparameterized DNNs.

This paper attempts to extend analytical results pertaining to the loss surface of linear networks to a nonlinear network with a single hidden ReLU layer.  Unfortunately though, at this point I feel that the theoretical results, which constitute the majority of the paper, are of limited novelty and/or significance.  However, I still remain very open to counterarguments to this opinion and the points raised below.
First, I don't believe that Lemma 2.2 is precisely true, at least as currently stated.  In particular, it would appear that L_f could have a differentiable local minima that is only a saddle point in L_gA.  For example, if there is a differentiable valley in L_f that terminates on the boundary of an activation region, then this phenomena could occur, since a local-minima-creating boundary in L_f might just lead to a saddle point in L_gA.  Regardless, the basic premise of this result is quite straightforward anyway.
Turning to Lemma 2.3 and 2.4, I don't understand the relevance of these results.  Where are they needed later or applied?  Additionally, Theorem 2.5 is very related to results already proven for linear networks in earlier work (Kawaguchi, 2016), so there is little novelty here.
There also seem to be issues with Corollary 2.7, which as an aggregation result can be viewed as the main contribution of the paper.  Part (1) of this corollary is obvious.  Part (2) depends on Lemma 2.2, which as stated previously may be problematic.  Most seriously though, Part (3) only considers critical points (i.e., derivative equal to zero), not local minima occurring at non-differentiable locations.  To me this greatly mutes the value of this result, and the contribution of the paper overall, because local minimum are *very* likely to occur on the boundary between activation regions at non-differentiable points (e.g. as in Figure 2).  I therefore don't understand the utility of only considering the differentiable local minima.
Overall though, the main point that within areas of fixed activation the network behaves much like a linear network (with all local minima also global minima when constrained within each region), is not especially noteworthy, because it provides no pathway for comparing minima from different activation regions, which is the main problem to begin with.
Beyond this, the paper makes a few less-technical observations regarding bad local minima.  For example, in Section 3.1 the argument is made that the linear region created when all activations are equal to one, will have a local minimum, and this minimum might be suboptimal.  However, these arguments pertain to the surrogate function L_gA, and if the minima to L_gA occurs on the boundary to another activation region, then this solution might not be a local minima to L_f, the real objective we care about.  Am I missing something here?
As for Section 4.2, the paper needs to do a better job of explaining exactly what is been shown in Table 2.  I can maybe guess, but it is not at all clear what the accuracy percentage is referring to, nor precisely how rich and random minima are computed.  Also, the assumption that P(a = 1) = P(a = 0) = 0.5 is not very realistic, although admittedly this type of simplification is sometimes adopted in the literature.
Minor comment:
* Near the beginning in the introduction, it is claimed that "the vanishing gradient problem has been solved by using rectified linear units."  This is not actually true, and portends problematic claims later in the paper.


PARAMETRIC INFORMATION BOTTLENECK TO OPTIMIZE STOCHASTIC NEURAL NETWORKS
In this paper, we present a layer-wise learning of stochastic neural networks (SNNs) in an information-theoretic perspective. In each layer of an SNN, the compression and the relevance are defined to quantify the amount of information that the layer contains about the input space and the target space, respectively. We jointly optimize the compression and the relevance of all parameters in an SNN to better exploit the neural network's representation. Previously, the Information Bottleneck (IB) framework (Tishby et al. (1999)) extracts relevant information for a target variable. Here, we propose Parametric Information Bottleneck (PIB) for a neural network by utilizing (only) its model parameters explicitly to approximate the compression and the relevance. We show that, the PIB framework can be considered as an extension of the maximum likelihood estimate (MLE) principle to every layer level. We also show that, as compared to the MLE principle, PIB : (i) improves the generalization of neural networks in classification tasks, (ii) is more efficient to exploit a neural network's representation by pushing it closer to the optimal information-theoretical representation in a faster manner.

This paper proposes a learning method (PIB) based on the information bottleneck framework.
PIB pursues the very natural intuition outlined in the information bottleneck literature: hidden layers of deep nets compress the input X while maintaining sufficient information to predict the output Y.
It should be noted that the limitations of the IB for deep learning are currently under heavy discussion on OpenReview.
Optimizing the PIB objective is intractable and the authors propose an approximation that applies to binary valued stochastic networks.
They use a variational bound to deal with the relevance term, I(Z_l,Y), and  Monte Carlo sampling to deal with the layer-by-layer compression term, I(Z_l,Z_{l+1}).
They present results on MNIST aiming to demonstrate that using PIBs improves generalization and training speed.
This is a timely and interesting topic. I enjoyed learning about the authors’ proposed approach to a practical learning method based on the information bottleneck. However, the writing made it challenging and the experimental protocol raised some serious questions. In summary, I think the paper needs very careful editing for grammar and language and, more importantly, it needs solid experiments before it’s ready for publication. When that is done it would make an exciting contribution to the community. More details follow.
Comments:
1. All architectures and objectives (both classic and PIB-based) are trained using a single, fixed learning rate (LR). In my opinion, this is a red flag. The PIB objective is new and different to the other objectives. Do all objectives happen to yield their best performance under the same LR? Maybe so, but we won’t know unless the experimental protocol prescribes a sufficient range of LRs for each architecture. In light of this, the fact that SFNN is given extra epochs in Figure 4 does not mean much.
2. The batch size for MNIST classification is unusually low (8). Common batch sizes range from 64 to 1K (typically >= 128). Why did the authors make this choice? Is 8 good for architectures A through E?
3. On a related note, the authors only seem to report results from a single random seed (ie. deterministic architectures are trained exactly once). I would like to see results from a few different random seeds. As a result of comments 1,2,3, even though I do believe in the merit of the intuition pursued and the techniques proposed, I am not convinced about the main claim of the paper. In particular, the experiments are not rigorous enough to give serious evidence that PIBs improve generalization and training speed.
4. The paper needs some careful editing both for language (cf. following point) but also notation. The authors use notation p_D() in eqn (12) without defining it. My best guess is that it is the same as p_u(), the underlying data distribution, but makes parsing the paper hard. Finally there are a few steps that are not explained: for example, no justification is given for the inequality in eqn (13).
5. Language: the paper needs some careful editing to correct numerous language/grammar issues. At times it is detrimental to understanding. For example I had to read the text leading up to eqn (8) a number of times.
6. There is no discussion of computational complexity and wall-clock time comparisons. To be clear, I think that even if the proposed approach were to be slower than the state of the art it would still be very interesting. However, there should be some discussion and reporting of that aspect as well.
Minor comments and questions:
7. Mutual information is typically typeset using a semicolon instead of a comma, eg. I(X;Z).
8. Why is the mutual information in Figure 3 so low? Are you perhaps using natural logarithms to estimate and plot I(Z;Y)? If this is base-2 logarithms I would expect a value close to 1.


We show how discrete objects can be learnt in an unsupervised fashion from pixels, and how to perform reinforcement learning using this object representation. More precisely, we construct a differentiable mapping from an image to a discrete tabular list of objects, where each object consists of a differentiable position, feature vector, and scalar presence value that allows the representation to be learnt using an attention mechanism. Applying this mapping to Atari games, together with an interaction net-style architecture for calculating quantities from objects, we construct agents that can play Atari games using objects learnt in an unsupervised fashion. During training, many natural objects emerge, such as the ball and paddles in Pong, and the submarine and fish in Seaquest. This gives the first reinforcement learning agent for Atari with an interpretable object representation, and opens the avenue for agents that can conduct objectbased exploration and generalization.

This paper learns to construct masks and feature representations from an input image, in order to represent objects. This is applied to the relatively simple domain of Atari games video input (compared to natural images). The paper is completely inadequate in respect to related work; it re-invents known techniques like non-maximum suppression and matching for tracking; fails to learn convincing objects according to visual inspection; and fails to compare with earlier methods for these tasks. (The comment above about re-invention is the most charitable intepretation -- the worst case would be using these ideas without citation.)
1) The related work section is outrageous, containing no references before 2016.  Do the authors think researchers never tried to do this task before then? This is the bad side of the recent deep nets hype, and ICLR is particularly susceptible to this. Examples include
@article{wang-adelson-94,
  author        = "Wang,  J. Y. A. and Adelson, E. H.",
  title         = {{Representing Moving Images with Layers}},
  journal       = {{IEEE Transactions on Image Processing}},
  year          = "1994",
  volume        = "3(5)",
  pages         = {625-638}
}
see http://persci.mit.edu/pub_pdfs/wang_tr279.pdf
and
@article{frey-jojic-03,
   author    = {Frey, B. J. and Jojic, N.},
   title     = {{Transformation Invariant Clustering Using the EM Algorithm}},
   journal   = {IEEE Trans Pattern Analysis and Machine Intelligence},
   year      = {2003},
   volume    = {25(1)},
   pages     = {1-17}
}
where mask and appearances for each object of interest are learned. There is a literature which follows on from the F&J paper.  The methods used in Frey & Jojic are different from what is proposed in the paper, but there needs to be comparisons.
The AIR paper also contains references to relevant previous work.
2) p 3 center -- this seems to be reinventing non-maximum suppression
3) p 4 eq 3 and sec 3.2 -- please justify *why* it makes sense to use
the concrete transform.  Can you explain better (e.g. in the supp mat)
the effect of this for different values of q_i?
4) Sec 3.5 Matching objects in successive frames using the Hungarian
algorithm is also well known, e.g. it is in the matlab function
assignDetectionsToTracks .
5) Overall: in this paper the authors come up with a method for learning objects from Atari games video input. This is a greatly restricted setting compared to real images. The objects learned as shown in Appendix A are quite unconvincing, e.g. on p 9. For example for Boxing why are the black and white objects broken up into 3 pieces, and why do they appear coloured in col 4?
Also the paper lacks comparisons to other methods (including ones from before 2016) which have tackled this problem.
It may be that the methods in this paper can outperform previous ones -- that would be interesting, but it would need a lot of work to address the issues raised above.
Text corrections:
p 2 "we are more precise" -> "we give more details"
p 3 and p 2 -- local maximum (not maxima) for a single maximum.  [occurs many times]