An ensemble of neural networks is known to be more robust and accurate than an individual network, however usually with linearly-increased cost in both training and testing. In this work, we propose a two-stage method to learn Sparse Structured Ensembles (SSEs) for neural networks. In the first stage, we run SG-MCMC with group sparse priors to draw an ensemble of samples from the posterior distribution of network parameters. In the second stage, we apply weight-pruning to each sampled network and then perform retraining over the remained connections. In this way of learning SSEs with SG-MCMC and pruning, we not only achieve high prediction accuracy since SG-MCMC enhances exploration of the modelparameter space, but also reduce memory and computation cost significantly in both training and testing of NN ensembles. This is thoroughly evaluated in the experiments of learning SSE ensembles of both FNNs and LSTMs. For example, in LSTM based language modeling (LM), we obtain 21% relative reduction in LM perplexity by learning a SSE of 4 large LSTM models, which has only 30% of model parameters and 70% of computations in total, as compared to the baseline large LSTM LM. To the best of our knowledge, this work represents the first methodology and empirical study of integrating SG-MCMC, group sparse prior and network pruning together for learning NN ensembles.

In this paper, the authors present a new framework for training ensemble of neural networks. The approach is based on the recent scalable MCMC methods, namely the stochastic gradient Langevin dynamics.
The paper is overall well-written and ideas are clear. The main contributions of the paper, namely using SG-MCMC methods within deep learning, and then increasing the computational efficiency by group sparsity+pruning are valuable and can have a significant impact in the domain. Besides, the proposed approach is more elegant the competing ones, while still not being theoretically justified completely.
I have the following minor comments:
1) The authors mention that retraining significantly improves the performance, even without pruning. What is the explanation for this? If there is no pruning, I would expect that all the samples would converge to the same minimum after retraining. Therefore, the reason why retraining improves the performance in all cases is not clear to me.
2) The notation |\theta_g| is confusing, the authors should use a different symbol.
3) After section 4, the language becomes quite informal sometimes, the authors should check the sentences once again.
4) The results with SGD (1 model) + GSP + PR should be added in order to have a better understanding of the improvements provided by the ensemble networks.
5) Why does the performance get worse "obviously" when the pruning is 95% and why is it not obvious when the pruning is 90%?
6) There are several typos
pg7: drew -> drawn
pg7: detail -> detailed
pg7: changing -> challenging
pg9: is strongly depend on -> depends on
pg9: two curve -> two curves


Published as a conference paper at ICLR 2018 NATURAL LANGUAGE INFERENCE OVER INTERACTION SPACE
Natural Language Inference (NLI) task requires an agent to determine the logical relationship between a natural language premise and a natural language hypothesis. We introduce Interactive Inference Network (IIN), a novel class of neural network architectures that is able to achieve high-level understanding of the sentence pair by hierarchically extracting semantic features from interaction space. We show that an interaction tensor (attention weight) contains semantic information to solve natural language inference, and a denser interaction tensor contains richer semantic information. One instance of such architecture, Densely Interactive Inference Network (DIIN), demonstrates the state-of-the-art performance on large scale NLI copora and large-scale NLI alike corpus. It's noteworthy that DIIN achieve a greater than 20% error reduction on the challenging Multi-Genre NLI (MultiNLI; ) dataset with respect to the strongest published system.

This paper proposes Densely Interactive Inference Network to solve recognizing textual entailment via extracting a semantic feature from interaction tensor end-to-end. Their results show that this model has better performance than others.
Even though the results of this paper is interesting, I have the problem with paper writing and motivation for their architecture:
- Paper pages are well beyond 8-page limits for ICLR. The paper should be 8-pages + References. This paper has 11 pages excluding the references.
-  The introduction text in the 2nd page doesn't have smooth flow and sometimes hard to follow.
-  In my view section, 3.1 is redundant and text in section 3.2 can be improved
-  Encoding layer in section 3.2 is really hard to follow in regards to equations and naming e.g p_{itr att} and why choose \alpha(a,b,w)?
-  Encoding layer in section 3.2, there is no motivation why it needs to use fuse gate.
-  Feature Extraction Layer is very confusing again. What is FSDR or TSDR?
-  Why the paper uses Eq. 8? the intuition behind it?
-  One important thing which is missing in this paper, I didn't understand what is the motivation behind using each of these components? and how each of these components is selected?
- How long does it take to train this network? Since it needs to works with other models (GLOV+ char features + POS tagging,..), it requires lots of effort to set up this network.
Even though the paper outperforms others, it would be useful to the community by providing the motivation and intuition why each of these components was chosen. This is important especially for this paper because each layer of their architecture uses multiple components, i.e. embedding layer [Glov+ Character Features + Syntactical features].  In my view, having just good results are not enough and will not guarantee a publication in ICLR, the paper should be well-written and well-motivated in order to be useful for the future research and the other researchers.
In summary, I don't think the paper is ready yet and it needs significant revision.
---------------------------------------------------------------------------------------------------------------------------------------------------------------
---------------------------------------------------------------------------------------------------------------------------------------------------------------
Comments after the rebuttal and revision :
I'd like thanks the authors for the revision and their answers.
Here are my comments after reading the revised version and considering the rebuttal:
- It is fair to say that the paper presentation is much better now. That said I am still having issues with 11 pages.
- The authors imply on page 2, end of paragraph 5,  that this is the first work that shows attention weight contains rich semantic and previous works are used attention merely as a medium for alignment.  Referring to the some of the related works (cited in this paper), I am not sure this is a correct statement.
- The authors claim to introduce a new class of architectures for NLI and generability of for this problem. In my view, this is a very strong statement and unsupported in the paper, especially considering ablation studies (table 5). In order for the model to show the best performance, all these components should come together. I am not sure why this method can be considered a class of architecture and why not just a new model?
some other comments:
- In page 4, the citation is missing for highway networks
- Page 5, equation 1, the parenthesis should close after \hat{P}_j.
Since the new version has been improved, I have increased my review score.  However, I'm still not convinced that this paper would be a good fit at ICLR given novelty and contribution.


INTERACTIVE GROUNDED LANGUAGE ACQUISITION AND GENERALIZATION IN A 2D WORLD
We build a virtual agent for learning language in a 2D maze-like world. The agent sees images of the surrounding environment, listens to a virtual teacher, and takes actions to receive rewards. It interactively learns the teacher's language from scratch based on two language use cases: sentence-directed navigation and question answering. It learns simultaneously the visual representations of the world, the language, and the action control. By disentangling language grounding from other computational routines and sharing a concept detection function between language grounding and prediction, the agent reliably interpolates and extrapolates to interpret sentences that contain new word combinations or new words missing from training sentences. The new words are transferred from the answers of language prediction. Such a language ability is trained and evaluated on a population of over 1.6 million distinct sentences consisting of 119 object words, 8 color words, 9 spatial-relation words, and 50 grammatical words. The proposed model significantly outperforms five comparison methods for interpreting zero-shot sentences. In addition, we demonstrate human-interpretable intermediate outputs of the model in the appendix.

[Overview]
In this paper, the authors proposed a unified model for combining vision, language, and action. It is aimed at controlling an agent in a virtual environment to move to a specified location in a 2D map, and answer user's questions as well. To address this problem, the authors proposed an explicit grounding way to connect the words in a sentence and spatial regions in the images. Specifically, By this way, the model could exploit the outputs of concept detection module to perform the actions and question answering as well jointly. In the experiments, the authors compared with several previous attention methods to show the effectiveness of the proposed concept detection module and demonstrated its superiority on several configurations, including in-domain and out-of-domain cases.
[Strengths]
1. I think this paper proposed interesting tasks to combine the vision, language, and actions. As we know, in a realistic environment, all three components are necessary to complete a complex tasks which need the interactions with the physical environments. The authors should release the dataset to prompt the research in this area.
2. The authors proposed a simple method to ground the language on visual input. Specifically, the authors grounded each word in a sentence to all locations of the visual map, and then perform a simple concept detection upon it. Then, the model used this intermediate representation to guide the navigation of agent in the 2D map and visual question answering as well.
3. From the experiments, it is shown that the proposed model outperforms several baseline methods in both normal tasks and out-of-domain ones. According to the visualizations, the interpreter could generate meaningful attention map given a textual query.
[Weakness]
1. The definition of explicit grounding is a bit misleading. Though the grounding or attention is performed for each word at each location of the visual map. It is a still kind of soft-attention, except that is performed for each word in a sentence. As far as I know, this has been done in several previous works, such as: (a). Hierarchical question-image co-attention for visual question answering (https://scholar.google.com/scholar?oi=bibs&cluster=15146345852176060026&btnI=1&hl=en). Lu et al. NIPS 2016. (b). Graph-Structured Representations for Visual Question Answering. Teney et al. arXiv 2016. At most recent, we have seen some more explicit way for visual grounding like: (c). Bottom-up and top-down attention for image captioning and VQA (https://arxiv.org/abs/1707.07998). Anderson et al. arXiv 2017.
2. Since the model is aimed at grounding the language on the vision based on interactions, it is worth to show how well the final model could ground the text words to each of the visual objects. Say, show the affinity matrix between the words and the objects to indicate the correlations.
[Summary]
I think this is a good paper which integrates vision, language, and actions in a virtual environment. I would foresee more and more works will be devoted to this area, considering its close connection to our daily life. To address this problem, the authors proposed a simple model to ground words on visual signals, which prove to outperform previous methods, such as CA, SAN, etc. According to the visualization, the model could attend the right region of the image for finishing a navigation and QA task. As I said, the authors should rephrase the definition of explicit grounding, to make it clearly distinguished with the previous work I listed above. Also, the authors should definitely show the grounding attention results of words and visual signal jointly, i.e., showing them together in one figure instead of separately in Figure 9 and Figure 10.


Published as a conference paper at ICLR 2018 POLICY OPTIMIZATION BY GENETIC DISTILLATION
Genetic algorithms have been widely used in many practical optimization problems. Inspired by natural selection, operators, including mutation, crossover and selection, provide effective heuristics for search and black-box optimization. However, they have not been shown useful for deep reinforcement learning, possibly due to the catastrophic consequence of parameter crossovers of neural networks. Here, we present Genetic Policy Optimization (GPO), a new genetic algorithm for sample-efficient deep policy optimization. GPO uses imitation learning for policy crossover in the state space and applies policy gradient methods for mutation. Our experiments on MuJoCo tasks show that GPO as a genetic algorithm is able to provide superior performance over the state-of-the-art policy gradient methods and achieves comparable or higher sample efficiency.

This is a highly interesting paper that proposes a set of methods that combine ideas from imitation learning, evolutionary computation and reinforcement learning in a novel way. It combines the following ingredients:
a) a population-based setup for RL
b) a pair-selection and crossover operator
c) a policy-gradient based “mutation” operator
d) filtering data by high-reward trajectories
e) two-stage policy distillation
In its current shape it has a couple of major flaws (but those can be fixed during the revision/rebuttal period):
(1) Related work. It is presented in a somewhat ahistoric fashion. In fact, ideas for evolutionary methods applied to RL tasks have been widely studied, and there is an entire research field called “neuroevolution” that specifically looks into which mutation and crossover operators work well for neural networks. I’m listing a small selection of relevant papers below, but I’d encourage the authors to read a bit more broadly, and relate their work to the myriad of related older methods. Ideally, a more reasonable form of parameter-crossover (see references) could be compared to -- the naive one is too much of a straw man in my opinion. To clarify: I think the proposed method is genuinely novel, but a bit of context would help the reader understand which aspects are and which aspects aren’t.
(2) Ablations. The proposed method has multiple ingredients, and some of these could be beneficial in isolation: for example a population of size 1 with an interleaved distillation phase where only the high-reward trajectories are preserved could be a good algorithm on its own. Or conversely, GPO without high-reward filtering during crossover. Or a simpler genetic algorithm that just preserves the kills off the worst members of the population, and replaces them by (mutated) clones of better ones, etc.
(3) Reproducibility. There are a lot of details missing; the setup is quite complex, but only partially described. Examples of missing details are: how are the high-reward trajectories filtered? What is the total computation time of the different variants and baselines? The x-axis on plots, does it include the data required for crossover/Dagger? What are do the shaded regions on plots indicate? The loss on \pi_S should be made explicit. An open-source release would be ideal.
Minor points:
- naively, the selection algorithm might not scale well with the population size (exhaustively comparing all pairs), maybe discuss that?
- the filtering of high-reward trajectories is what estimation of distribution algorithms [2] do as well, and they have a known failure mode of premature convergence because diversity/variance shrinks too fast. Did you investigate this?
- for Figure 2a it would be clearer to normalize such that 1 is the best and 0 is the random policy, instead of 0 being score 0.
- the language at the end of section 3 is very vague and noncommittal -- maybe just state what you did, and separately give future work suggestions?
- there are multiple distinct metrics that could be used on the x-axis of plots, namely: wallclock time, sample complexity, number of updates. I suspect that the results will look different when plotted in different ways, and would enjoy some extra plots in the appendix. For example the ordering in Figure 6 would be inverted if plotting as a function of sample complexity?
- the A2C results are much worse, presumably because batchsizes are different? So I’m not sure how to interpret them: should they have been run for longer? Maybe they could be relegated to the appendix?
References:
[1] Gomez, F. J., & Miikkulainen, R. (1999). Solving non-Markovian control tasks with neuroevolution.
[2] Larranaga, P. (2002). A review on estimation of distribution algorithms.
[3] Stanley, K. O., & Miikkulainen, R. (2002). Evolving neural networks through augmenting topologies.
[4] Igel, C. (2003). Neuroevolution for reinforcement learning using evolution strategies.
[5] Hausknecht, M., Lehman, J., Miikkulainen, R., & Stone, P. (2014). A neuroevolution approach to general atari game playing.
[6] Gomez, F., Schmidhuber, J., & Miikkulainen, R. (2006). Efficient nonlinear control through neuroevolution.
Pros:
- results
- novelty of idea
- crossover visualization, analysis
- scalability
Cons:
- missing background
- missing ablations
- missing details
[after rebuttal: revised the score from 7 to 8]


Published as a conference paper at ICLR 2018 DEEP NEURAL NETWORKS AS GAUSSIAN PROCESSES
It has long been known that a single-layer fully-connected neural network with an i.i.d. prior over its parameters is equivalent to a Gaussian process (GP), in the limit of infinite network width. This correspondence enables exact Bayesian inference for infinite width neural networks on regression tasks by means of evaluating the corresponding GP. Recently, kernel functions which mimic multi-layer random neural networks have been developed, but only outside of a Bayesian framework. As such, previous work has not identified that these kernels can be used as covariance functions for GPs and allow fully Bayesian prediction with a deep neural network. In this work, we derive the exact equivalence between infinitely wide deep networks and GPs. We further develop a computationally efficient pipeline to compute the covariance function for these GPs. We then use the resulting GPs to perform Bayesian inference for wide deep neural networks on MNIST and CIFAR-10. We observe that trained neural network accuracy approaches that of the corresponding GP with increasing layer width, and that the GP uncertainty is strongly correlated with trained network prediction error. We further find that test performance increases as finite-width trained networks are made wider and more similar to a GP, and thus that GP predictions typically outperform those of finite-width networks. Finally we connect the performance of these GPs to the recent theory of signal propagation in random neural networks.

Neal (1994) showed that a one hidden layer Bayesian neural network, under certain conditions, converges to a Gaussian process as the number of hidden units approaches infinity. Neal (1994) and Williams (1997) derive the resulting kernel functions for such Gaussian processes when the neural networks have certain transfer functions.
Similarly, the authors show an analogous result for deep neural networks with multiple hidden layers and an infinite number of hidden units per layer, and show the form of the resulting kernel functions. For certain transfer functions, the authors perform a numerical integration to compute the resulting kernels. They perform experiments on MNIST and CIFAR-10, doing classification by scaled regression.
Overall, the work is an interesting read, and a nice follow-up to Neal’s earlier observations about 1 hidden layer neural networks. It combines several insights into a nice narrative about infinite Bayesian deep networks. However, the practical utility, significance, and novelty of this work -- in its current form -- are questionable, and the related work sections, analysis, and experiments should be significantly extended.
In detail:
(1) This paper misses some obvious connections and references, such as
* Krauth et. al (2017): “Exploring the capabilities and limitations of Gaussian process models” for recursive kernels with GPs.
* Hazzan & Jakkola (2015): “Steps Toward Deep Kernel Methods from Infinite Neural Networks” for GPs corresponding to NNs with more than one hidden layer.
* The growing body of work on deep kernel learning, which “combines the inductive biases and representation learning abilities of deep neural networks with the non-parametric flexibility of Gaussian processes”. E.g.: (i) “Deep Kernel Learning” (AISTATS 2016); (ii) “Stochastic Variational Deep Kernel Learning” (NIPS 2016); (iii) “Learning Scalable Deep Kernels with Recurrent Structure” (JMLR 2017).
These works should be discussed in the text.
(2) Moreover, as the authors rightly point out, covariance functions of the form used in (4) have already been proposed. It seems the novelty here is mainly the empirical exploration (will return to this later), and numerical integration for various activation functions. That is perfectly fine -- and this work is still valuable. However, the statement “recently, kernel functions for multi-layer random neural networks have been developed, but only outside of a Bayesian framework” is incorrect. For example, Hazzan & Jakkola (2015) in “Steps Toward Deep Kernel Methods from Infinite Neural Networks” consider GP constructions with more than one hidden layer. Thus the novelty of this aspect of the paper is overstated.
See also comment [*] later on the presentation. In any case, the derivation for computing the covariance function (4) of a multi-layer network is a very simple reapplication of the procedure in Neal (1994). What is less trivial is estimating (4) for various activations, and that seems to the major methodological contribution.
Also note that multidimensional CLT here is glossed over. It’s actually really unclear whether the final limit will converge to a multidimensional Gaussian with that kernel without stronger conditions.  This derivation should be treated more thoroughly and carefully.
(3) Most importantly, in this derivation, we see that the kernels lose the interesting representations that come from depth in deep neural networks. Indeed, Neal himself says that in the multi-output settings, all the outputs become uncorrelated. Multi-layer representations are mostly interesting because each layer shares hidden basis functions. Here, the sharing is essentially meaningless, because the variance of the weights in this derivation shrinks to zero.
In Neal’s case, the method was explored for single output regression, where the fact that we lose this sharing of basis functions may not be so restrictive. However, these assumptions are very constraining for multi-output classification and also interesting multi-output regressions.
[*]: Generally, in reading the abstract and introduction, we get the impression that this work somehow allows us to use really deep and infinitely wide neural networks as Gaussian processes, and even without the pain of training these networks. “Deep neural networks without training deep networks”. This is not an accurate portrayal. The very title “Deep neural networks as Gaussian processes” is misleading, since it’s not really the deep neural networks that we know and love. In fact, you lose valuable structure when you take these limits, and what you get is very different than a standard deep neural network. In this sense, the presentation should be re-worked.
(4) Moreover, neural networks are mostly interesting because they learn the representation. To do something similar with GPs, we would need to learn the kernel. But here, essentially no kernel learning is happening. The kernel is fixed.
(5) Given the above considerations, there is great importance in understanding the practical utility of the proposed approach through a detailed empirical evaluation. In other words, how structured is this prior and does it really give us some of the interesting properties of deep neural networks, or is it mostly a cute mathematical trick?
Unfortunately, the empirical evaluation is very preliminary, and provides no reassurance that this approach will have any practical relevance:
(i) Directly performing regression on classification problems is very heuristic and unnecessary.
(ii) Given the loss of dependence between neurons in this approach, it makes sense to first explore this method on single output regression, where we will likely get the best idea of its useful properties and advantages.
(iii) The results on CIFAR10 are very poor. We don’t need to see SOTA performance to get some useful insights in comparing for example parametric vs non-parametric, but 40% more error than SOTA makes it very hard to say whether any of the observed patterns hold weight for more competitive architectural choices.
A few more minor comments:
(i) How are you training a GP exactly on 50k training points? Even storing a 50k x 50k matrix requires about 20GB of RAM. Even with the best hardware, computing the marginal likelihood dozens of times to learn hyperparameters would be near impossible. What are the runtimes?
(ii) "One benefit in using the GP is due to its Bayesian nature, so that predictions have uncertainty estimates (Equation (9)).”  The main benefit of the GP is not the uncertainty in the predictions, but the marginal likelihood which is useful for kernel learning.


Regularization is one of the crucial ingredients of deep learning, yet the term regularization has various definitions, and regularization methods are often studied separately from each other. In our work we present a novel, systematic, unifying taxonomy to categorize existing methods. We distinguish methods that affect data, network architectures, error terms, regularization terms, and optimization procedures. We identify the atomic building blocks of existing methods, and decouple the assumptions they enforce from the mathematical tools they rely on. We do not provide all details about the listed methods; instead, we present an overview of how the methods can be sorted into meaningful categories and sub-categories. This helps revealing links and fundamental similarities between them. Finally, we include practical recommendations both for users and for developers of new regularization methods.

This paper is unusual in that it is more of a review than contributing novel knowledge. It considers a taxonomy of all the ways that machine learning (mostly deep learning) methods can achieve a form of regularization.
Unfortunately, it starts with a definition of regularization ('making the model generalize better') which I believe misses the point which was made in Goodfellow et al 2016 ('intend to improve test error but not necessarily training error'), i.e., that we would like to separate as much as possible the regularization effects from the optimization effect. Indeed, under the definition proposed here, any improvement in the optimizer could be considered like a regularizer, so long as we are not in the overfitting regime. That does not sound right to me.
There are several places where the authors make TOO STRONG STATEMENTS, taking for truth what are simply beliefs with no strong supporting evidence (at least published). This is not good for a review and when making recommendations.
The other weakness I estimate in this paper is that I did not get a sense that the taxonomy really helped us (me at least) to get insight into the different mentions being cited. Besides the obvious proposal to combine ideas to write new papers (but we did not need that paper to figure that out) I did not find much meat in the 'future directions' section.
However, I that except in a few places the understand of the field displayed by the authors is pretty good and, with correction, could serve as a useful reference for students of deep learning. The recommendations were reasonable although lacking empirical support (or pointers to the literature), so I would take them somewhat carefully, more as the current 'group think' than ground truth.
Finally, here a few minor points which could be fixed.
Eq. 1: in typical DL, minimization is approximate, not exact, so the proposed formalism does not reflect reality.
Eq. 4: in many cases, the noise is not added (e.g. dropout), so that should be clarified there.
page 3, first bullet of 'Effect on the data representation': not clear, may want to give translations as an example of such transformations,.
page 8, activation functions: the ReLU is actually older than the cited papers, it was used by computational neuroscientists a long time ago. Jarrett 2009 did not use the ReLU but an absolute-value rectifier and it was Glorot 2011 who showed that the ReLU really kicked ass for deeper networks. Nair 2010 used the ReLU in a very different context (RBMs), not really feedforward multi-layer networks where it shines now.
In that same section (and probably elsewhere) there are TOO STRONG STATEMENTS, e.g., the "facts" mentioned are not facts but merely folk belief, as far as I know, and I would like to see well-done supporting evidence before treating those as facts. Note that approximating the sigmoid precisely would require many ReLUs!
page 8: it is not clear how multi-task learning fits under the 'architecture' formalism provided at the beginning of section 4.
section 7 (page 10): there is earlier work on the connection between early stopping and L2 regularization, at least dating back to Ronan Collobert's PhD thesis (with neural nets), probably earlier for linear systems.


We propose a metric-learning framework for computing distance-preserving maps that generate low-dimensional embeddings for a certain class of manifolds. We employ Siamese networks to solve the problem of least squares multidimensional scaling for generating mappings that preserve geodesic distances on the manifold. In contrast to previous parametric manifold learning methods we show a substantial reduction in training effort enabled by the computation of geodesic distances in a farthest point sampling strategy. Additionally, the use of a network to model the distance-preserving map reduces the complexity of the multidimensional scaling problem and leads to an improved non-local generalization of the manifold compared to analogous non-parametric counterparts. We demonstrate our claims on point-cloud data and on image manifolds and show a numerical analysis of our technique to facilitate a greater understanding of the representational power of neural networks in modeling manifold data.

The authors argue that the spectral dimensionality reduction techniques are too slow, due to the complexity of computing the eigenvalue decomposition, and that they are not suitable for out-of-sample extension. They also note the limitation of neural networks, which require huge amounts of data to properly learn the data structure. The authors therefore propose to first sub-sample the data and afterwards to learn an MDS-like cost function directly with a neural network, resulting in a parametric framework.
The paper should be checked for grammatical errors, such as e.g. consistent use of (no) hyphen in low-dimensional (or low dimensional).
The abbreviations should be written out on the first use, e.g. MLP, MDS, LLE, etc.
In the introduction the authors claim that the complexity of parametric techniques does not depend on the number of data points, or that moving to parametric techniques would reduce memory and computational complexities. This is in general not true. Even if the number of parameters is small, learning them might require complex computations on the whole data set. On the other hand, even if the number of parameters is equal to the number of data points, the computations could be trivial, thus resulting in a complexity of O(N).
In section 2.1, the authors claim "Spectral techniques are non-parametric in nature"; this is wrong again. E.g. PCA can be formulated as MDS (thus spectral), but can be seen as a parametric mapping which can be used to project new words.
In section 2.2, it says "observation that the double centering...". Can you provide a citation for this?
In section 3, the authors propose they technique, which should be faster and require less data than the previous methods, but to support their claim, they do not perform an analysis of computational complexity. It is not quite clear from the text what the resulting complexity would be. With N as number of data points and M as number of landmarks, from the description on page 4 it seems the complexity would be O(N + M^2), but the steps 1 and 2 on page 5 suggest it would be O(N^2 + M^2). Unfortunately, it is also not clear what the complexity of previous techniques, e.g DrLim, is.
Figure 3, contrary to text, does not provide a visualisation to the sampling mechanism.
In the experiments section, can you provide a citation for ADAM and explain how the parameters were selected? Also, it is not meaningful to measure the quality of a visualisation via the MDS fit. There are more useful approaches to this task, such as the quality framework [*].
In figure 4a, x-axis should be "number of landmarks".
It is not clear why the equation 6 holds. Citation?
It is also not clear how exactly the equation 7 is evaluated. It says "By varying the number of layers and the number of nodes...", but the nodes and layer are not a part of the equation.
The notation for equation 8 is not explained.
Figure 6a shows visualisations by different techniques and is evaluated "by looking at it". Again, use [*].
[*] Lee, John Aldo ; Verleysen, Michel. Scale-independent quality criteria for dimensionality reduction. In: Pattern Recognition Letters, Vol. 31, no. 14, p. 2248-2257 (2010). doi:10.1016/j.patrec.2010.04.013.


Inspired by neurophysiological discoveries of navigation cells in the mammalian brain, we introduce the first deep neural network architecture for modeling Egocentric Spatial Memory (ESM). It learns to estimate the pose of the agent and progressively construct top-down 2D global maps from egocentric views in a spatially extended environment. During the exploration, our proposed ESM network model updates belief of the global map based on local observations using a recurrent neural network. It also augments the local mapping with a novel external memory to encode and store latent representations of the visited places based on their corresponding locations in the egocentric coordinate. This enables the agents to perform loop closure and mapping correction. This work contributes in the following aspects: first, our proposed ESM network provides an accurate mapping ability which is vitally important for embodied agents to navigate to goal locations.
In the experiments, we demonstrate the functionalities of the ESM network in random walks in complicated 3D mazes by comparing with several competitive baselines and state-of-the-art Simultaneous Localization and Mapping (SLAM) algorithms. Secondly, we faithfully hypothesize the functionality and the working mechanism of navigation cells in the brain. Comprehensive analysis of our model suggests the essential role of individual modules in our proposed architecture and demonstrates efficiency of communications among these modules. We hope this work would advance research in the collaboration and communications over both fields of computer science and computational neuroscience.

Significance of Contributions Unclear
The paper describes a neural network architecture for monocular SLAM that is argued to take inspiration from neuroscience. The architecture is comprised of four components: one that estimates egomotion (HDU) much like prediction in a filtering framework; one that fuses the current image into a local 2D metric map (BVU); one that detects loop closures (PCU); and one that integrates local maps (GU). These modules along with their associated representations are learned in an end-to-end fashion. The method is trained and evaluated on simulated grid environments and compared to two visual SLAM algorithms.
The contributions and significance of the paper are unclear. SLAM is arguably a solved problem at the scales considered here, with existing solutions capable of performing localization and mapping in large (city-scale), real-world environments. That aside, one can appreciate the merits of representation learning in the context of SLAM and a handful of neural network-based approaches to SLAM and the related problem of navigation have been proposed of-late. However, the paper doesn't do a sufficient job making the advantages of the proposed approach over these methods clear. Further, the paper emphasizes parallels to neuroscience models for navigation as being a contribution, however these similarities are largely hand wavy and one could argue that they also exist for the many other SLAM algorithms that perform prediction (as in HDU), local/global mapping (as in BVU and GU) and loop closure detection (as in PCU). More fundamentally, the proposed method does not appear to account for motion or measurement noise that are inherent in any SLAM problem and, related, does not attempt to model the uncertainty in the resulting map or pose estimates.
The paper evaluates the individual components of the architecture. The results suggest that the different modules are doing something reasonable, though the evaluation is rather limited (in terms of spatial scale) and a bit arbitrary (e.g., comparing local maps to the ground truth at a seemingly arbitrary 32s). The evaluation of the loop closure is limited to a qualitative measure and is therefore not convincing. The authors should quantitatively evaluate the performance of loop closure in terms of precision and recall (this is particularly important given effects of erroneous loop closure detections and the claims that the proposed method is robust). Meanwhile, it isn't clear that much can be concluded from the ablation studies as there is relatively little difference in MSE between the two ablated models.
Additional comments/questions:
* A stated advantage of this method over that of Gupta et al. is that the agent's motion is not assumed to be known. However, it isn't clear whether and how the model incorporates motion or measurement uncertainty, which is fundamental to any SLAM (or navigation) framework.
* Related, an important aspect of any SLAM algorithm is an explicit estimate of the uncertainty in the agent's pose and the map, however it doesn't seem that the proposed model attempts to express this uncertainty.
* The paper claims to estimate the agent's pose as it navigates, but it is not apparent how the pose is maintained beyond estimating egomotion by comparing the current image to a local map.
* Related, it is not clear how the method balances egomotion estimates and exteroceptive measurements (e.g, as are fused with traditional filtering frameworks). There are vague references to "eliminating discrepancies" when merging measurements, but it isn't clear what this actually means, whether the output is consistent, or how the advantages of egomotion estimation and measurements are balanced.
* The BVU module is stated as generating a "local" map, but it is not clear from the discussion what limits the resulting map to the area in the vicinity of the robot vs. the entire environment.
* It is not clear why previous data is transformed to the agent's reference frame as a result of motion vs. the more traditional approach of transforming the agent's pose to a global reference frame.
* The description of loop closure detection and the associated heuristics is confusing. For example, Section 3.4 states that the agent only considers positions that are distant from the most recent visited position as a means of avoiding trival loop closures, however Section 3.4 states that GU provides memory vectors near the current location for loop closure classification.
* The description of the GU module is confusing. How does spatial indexing deal with changes to the map (e.g., as a result of loop closures/measurement updates) or transformations to the robot's frame-of-reference? What are h, H, w, and W and how are they chosen?
* The architecture assumes a discrete (and course) action space, whereas actions are typically continuous. Have the authors tried regressing to continuous actions or experimenting with finer discretizations that are more suitable to real applications?
* It is not clear what is meant by the statement that the PU "learns to encode the representation of visited places".
* The means by which the architecture is trained is unclear. What is the loss that is optimized? How is the triplet loss (Eqn. 3) incorporated (e.g., is it weighted differently than other terms in the loss)?
* Section 3.2 states that the "agent has to learn to take actions to explore its surroundings", however it isn't apparent that the method reasons over the agent's policy. Indeed, this is an open area of research. Instead, the results section suggests that the agent acts randomly.
* Section 4.1 draws comparisons between HDU and Head Direction Cells, however the latter estimate location/orientation whereas the former (this method) predicts egomotion. While egomotion can be integrated to estimate pose (as is done in Fig 4), these are not the same thing.
* The authors are encouraged to tone down claims regarding parallels to navigation models from neuroscience as they are largely unjustified.
* The comparison to existing monocular SLAM baselines is surprising and the reviewer remains skeptical regarding the stated advantages of the proposed method. How much of this difference is a result of testing in simulation? It would be more convincing to compare performance in real-world environments, for which these baselines have proven effective.
* Figure 1: "Border" --> "Boundary"
* Figure 1: The camera image should also go to the BVU block
* Many of the citations are incorrectly not parenthesized
* The paper should be proof-read for grammatical errors


Published as a conference paper at ICLR 2018 DEEP VOICE 3: SCALING TEXT-TO-SPEECH WITH CONVOLUTIONAL SEQUENCE LEARNING
We present Deep Voice 3, a fully-convolutional attention-based neural textto-speech (TTS) system. Deep Voice 3 matches state-of-the-art neural speech synthesis systems in naturalness while training an order of magnitude faster. We scale Deep Voice 3 to dataset sizes unprecedented for TTS, training on more than eight hundred hours of audio from over two thousand speakers. In addition, we identify common error modes of attention-based speech synthesis networks, demonstrate how to mitigate them, and compare several different waveform synthesis methods. We also describe how to scale inference to ten million queries per day on a single GPU server.

This paper provides an overview of the Deep Voice 3 text-to-speech system. It describes the system in a fair amount of detail and discusses some trade-offs w.r.t. audio quality and computational constraints. Some experimental validation of certain architectural choices is also provided.
My main concern with this work is that it reads more like a tech report: it describes the workings and design choices behind one particular system in great detail, but often these choices are simply stated as fact and not really motivated, or compared to alternatives. This makes it difficult to tell which of these aspects are crucial to get good performance, and which are just arbitrary choices that happen to work okay.
As this system was clearly developed with actual deployment in mind (and not purely as an academic pursuit), all of these choices must have been well-deliberated. It is unfortunate that the paper doesn't demonstrate this. I think this makes the work less interesting overall to an ICLR audience. That said, it is perhaps useful to get some insight into what types of models are actually used in practice.
An exception to this is the comparison of "converters", model components that convert the model's internal representation of speech into waveforms. This comparison is particularly interesting because some of the results are remarkable, i.e. Griffin-Lim spectrogram inversion and the WORLD vocoder achieving very similar MOS scores in some cases (Table 2). I wish there would be more of that kind of thing in the paper. The comparison of attention mechanisms is also useful.
I'm on the fence as I think it is nice to get some insight into a practical pipeline which benefits from many current trends in deep learning research (autoregressive models, monotonic attention, ...), but I also feel that the paper is a bit meager when it comes to motivating all the architectural aspects. I think the paper is well written so I've tentatively recommended acceptance.
Other comments:
- The separation of the "decoder" and "converter" stage is not entirely clear to me. It seems that the decoder is trained to predict spectrograms autoregressively, but its final layer is then discarded and its hidden representation is then used as input to the converter stage instead? The motivation for doing this is unclear to me, surely it would be better to train everything end-to-end, including the converter? This seems like an unnecessary detour, what's the reasoning behind this?
- At the bottom of page 2 it is said that "the whole model is trained end-to-end, excluding the vocoder", which I think is an unfortunate turn of phrase. It's either end-to-end, or it isn't.
- In Section 3.3, the point of mixing of h_k and h_e is unclear to me. Why is this done?
- The gated linear unit in Figure 2a shows that speaker embedding information is only injected in the linear part. Has this been experimentally validated to work better than simpler mechanisms such as adding conditioning-dependent biases/gains?
- When the decoder is trained to do autoregressive prediction of spectrograms, is it autoregressive only in time, or also in frequency? I'm guessing it's the former, but this means there is an implicit independence assumption (the intensities in different frequency bins are conditionally independent, given all past timesteps). Has this been taken into consideration? Maybe it doesn't matter because the decoder is never used directly anyway, and this is only a "feature learning" stage of sorts?
- Why use the L1 loss on spectrograms?
- The recent work on Parallel WaveNet may allow for speeding up WaveNet when used as a vocoder, this could be worth looking into seeing as inference speed is used as an argument to choose different vocoder strategies (with poorer audio quality as a result).
- The title heavily emphasizes that this model can do multi-speaker TTS with many (2000) speakers, but that seems to be only a minor aspect that is only discussed briefly in the paper. And it is also something that preceding systems were already capable of (although maybe it hasn't been tested with a dataset of this size before). It might make sense to rethink the title to emphasize some of the more relevant and novel aspects of this work.
----
Revision: the authors have adequately addressed quite a few instances where I feel motivations / explanations were lacking, so I'm happy to increase my rating from 6 to 7. I think the proposed title change would also be a good idea.


Neural network-based systems can now learn to locate the referents of words and phrases in images, answer questions about visual scenes, and even execute symbolic instructions as first-person actors in partially-observable worlds. To achieve this so-called grounded language learning, models must overcome certain well-studied learning challenges that are also fundamental to infants learning their first words. While it is notable that models with no meaningful prior knowledge overcome these learning obstacles, AI researchers and practitioners currently lack a clear understanding of exactly how they do so. Here we address this question as a way of achieving a clearer general understanding of grounded language learning, both to inform future research and to improve confidence in model predictions.
For maximum control and generality, we focus on a simple neural network-based language learning agent trained via policy-gradient methods to interpret synthetic linguistic instructions in a simulated 3D world. We apply experimental paradigms from developmental psychology to this agent, exploring the conditions under which established human biases and learning effects emerge. We further propose a novel way to visualise and analyse semantic representation in grounded language learning agents that yields a plausible algorithmic account of the observed effects.

This paper presents an analysis of an agent trained to follow linguistic commands in a 3D environment.  The behaviour of the agent is analyzed by means of a set of "psycholinguistic" experiments probing what it learned, and by inspection of its visual component through an attentional mechanism.
On the positive side, it is nice to read a paper that focuses on understanding what an agent is learning. On the negative side, I did not get many new insights from the analyses presented in the study.
3 A situated language learning agent
I can't make up the chair from the refrigerator in the figure.
4.1 Word learning biases
This experiment shows that, when an agent is trained on shapes only, it will exhibit a shape bias when tested on new shapes and colors. Conversely, when it is exposed to colors only, it will have a color bias. When the training set is balanced, the agent shows a mild bias for the simpler color property. How is this interesting or surprising? The crucial question, here, would be whether, when an agent is trained in a naturalistic environment (i.e., where distributions of colors, shapes and other properties reflect those encountered by biological agents), it would show a human-like shape bias. This, however, is not addressed in the paper.
Minor comments about this section:
- Was there noise also in shape generation, or were all object instances identical?
- propensity to select o_2: rather o_1?
- I did not follow the paragraph starting with "This effect provides".
4.2 The problem of learning negation
I found this experiment very interesting.
Perhaps, the authors could be more explicit about the usage of negation here. The meaning of commands containing negation are, I think, conjunctions of the form "pick something and do not pick X" (as opposed to the more natural "do not pick X").
modifiation: modification
4.3 Curriculum learning
Perhaps the difference in curriculum effectiveness in language modeling vs grounded language learning simulations is due to the fact that the former operates on large amounts of natural data, where it's hard to define the curriculum, while the latter are typically grounded in toy worlds with a controlled language, where it's easier to construct the curriculum.
4.4 Processing and representation differences
There is virtually no discussion of what makes the naturalistic setup naturalistic, and thus it's not clear which conclusions we should derive from the corresponding experiments. Also, I don't see what we should learn from Figure 5 (besides the fact that in the controlled condition shapes are easier than categories). For the naturalistic condition, the current figure is misleading, since different classes contain different numbers of instances. It would be better to report proportions.
Concerning the attention analysis, it seems to me that all it's saying is that lower layers of a CNN detect lower-level properties such as colors, higher layers detect more complex properties, such as shapes characterizing objects. What is novel here?
Also, since introducing attention changes the architecture, shouldn't the paper report the learning behaviour of the attention-augmented network?
The explanation of the attention mechanism is dense, and perhaps could be aided by a diagram (in the supplementary materials?). I think the description uses "length" when "dimensional(ity)" is meant.
6. Supplementary material
It would be good to have an explicit description of the architecture, including number of layers of the various components, structure of the CNN, non-linearities, dimensionality of the layers, etc. (some of this information is inconsistently provided in the paper).
It's interesting that the encoder is actually a BOW model. This should be discussed in the paper, as it raises concerns about the linguistic interest of the controlled language that was used.
Table 3: indicates is: indicates if


Progress in probabilistic generative models has accelerated, developing richer models with neural architectures, implicit densities, and with scalable algorithms for their Bayesian inference. However, there has been limited progress in models that capture causal relationships, for example, how individual genetic factors cause major human diseases. In this work, we focus on two challenges in particular: How do we build richer causal models, which can capture highly nonlinear relationships and interactions between multiple causes? How do we adjust for latent confounders, which are variables influencing both cause and effect and which prevent learning of causal relationships? To address these challenges, we synthesize ideas from causality and modern probabilistic modeling. For the first, we describe implicit causal models, a class of causal models that leverages neural architectures with an implicit density. For the second, we describe an implicit causal model that adjusts for confounders by sharing strength across examples. In experiments, we scale Bayesian inference on up to a billion genetic measurements. We achieve state of the art accuracy for identifying causal factors: we significantly outperform existing genetics methods by an absolute difference of 15-45.3%.

In this paper, the authors propose to use the so-called implicit model to tackle Genome-Wide Association problem. The model can be viewed as a variant of Structural Equation Model. Overal the paper is interesting and relatively well-written but some important details are missing and way more experiments need to be done to show the effectiveness of the approach.
*  How do the authors call a variant to be associated with the phenotype (y)? More specifically, what is the distribution of the null hypothesis? Section D.3 in the appendix does not explain the hypothesis testing part well. This method models $x$ (genetic), $y$ (phenotype), and $z$ (confounder) but does not have a latent variable for the association. For example, there is no latent indicator variable (e.g., Spike-Slab models [1]) for each variant.  Did they do hypothesis testing separately after they fit the model? If so, this has double dipping problem because the data is used once to fit the model and again to perform statistical inference.
* In GWAS, a method resulting in high power with control of FP is favored. In traditional univariate GWAS, the false positive rate is controlled by genome-wide significant level (7e-8), Bonferroni correction or other FP control approaches.  Why Table 1 does not report FP? I need Table 1 to report the following: What is the power of this method if FPR is controlled(False Positive Rate < 0.05)? Also, the ROC curve for FPR<0.05 should be reported for all methods.
* I believe that authors did a good job in term of a survey of the available models for GWA from marginal regression to mixed effect model, etc. The authors account for typical confounders such as cryptic relatedness which I liked. However, I recommend the authors to be cautious calling the association detected by their method " a Causal Association." There are tons of research done to understand the causal effect of the genetic variants and this paper (and this venue) is not addressing those.  There are several ways for an associated variant to be non-causal and this paper does not even scratch the surface of that. For example, in many studies, discovering the causal SNPs means finding a genetic variant among the SNPs in LD of each other (so-called fine mapping). The LD-pruning procedure proposed in this paper does not help for that purpose.
* This approach jointly models the genetic variants and the phenotype (y). Let us assume that one can directly maximize the ML (ELBO maximizes a lower bound of ML). The objective function is disproportionally influenced by the genetic variants (x) than y because M is very large (  $\prod_{m=1}^M p(w) p(x|z,w,\phi)   >>   p(z) p(y|x,z,\theta) $  ). Effectively, the model focuses on the genetic variants, not by the disease. This is why multi-variate GWAS focuses on the conditional p(y|x,z) and not p(y,x,z). Nothing was shown in the paper that this focusing on p(y,x,z) is advantageous to p(y|x,z).
* In this paper, the authors use deep neural networks to model the general functional causal models. Since estimation of the causal effects is generally unidentifiable (Sprites 1993), I think using a general functional causal model with confounder modeling would have a larger chance to weaken the causal effects because the confounder part can also explain part of the causal influences. Is there a theoretical guarantee for the proposed method? Practically, how did the authors control the model complexity to avoid trivial solutions?
Minor
-------
* The idea of representing (conditional) densities by neural networks was proposed in the generative adversarial networks (GAN). In this paper, the authors represent the functional causal models by neural networks, which is very related to the representation used in GANs. The only difference is that GAN does not specify a causal interpretation. I suggest the authors add a short discussion of the relations to GAN.
* Previous methods on causal discovery rely on restricted functional causal models for identifiability results. They also use Gaussian process or multi-layer perceptron to model the functions implicitly, which can be consider as neural networks with one hidden layer. The sentence “These models typically focus on the task of causal discovery, and they assume fixed nonlinearities or smoothness which we relax using neural networks.” in the related work section is not appropriate.
[1] Scalable Variational Inference for Bayesian Variable Selection in Regression, and Its Accuracy in Genetic Association Studies


Published as a conference paper at ICLR 2018 N2N LEARNING: NETWORK TO NETWORK COMPRESSION VIA POLICY GRADIENT REINFORCEMENT LEARNING
While wider and deeper neural network architectures continue to advance the state-of-the-art for many computer vision tasks, real-world adoption of these networks is impeded by hardware and speed constraints. Conventional model compression methods attempt to address this problem by modifying the architecture manually or using pre-defined heuristics. Since the space of all reduced architectures is very large, modifying the architecture of a deep neural network in this way is a difficult task. In this paper, we tackle this issue by introducing a principled method for learning reduced network architectures in a data-driven way using reinforcement learning. Our approach takes a larger 'teacher' network as input and outputs a compressed 'student' network derived from the 'teacher' network. In the first stage of our method, a recurrent policy network aggressively removes layers from the large 'teacher' model. In the second stage, another recurrent policy network carefully reduces the size of each remaining layer. The resulting network is then evaluated to obtain a reward -a score based on the accuracy and compression of the network. Our approach uses this reward signal with policy gradients to train the policies to find a locally optimal student network. Our experiments show that we can achieve compression rates of more than 10× for models such as ResNet-34 while maintaining similar performance to the input 'teacher' network. We also present a valuable transfer learning result which shows that policies which are pre-trained on smaller 'teacher' networks can be used to rapidly speed up training on larger 'teacher' networks.

Summary:
The manuscript introduces a principled way of network to network compression, which uses policy gradients for optimizing two policies which compress a strong teacher into a strong but smaller student model. The first policy, specialized on architecture selection, iteratively removes layers, starting with architecture of the teacher model. After the first policy is finished, the second policy reduces the size of each layer by iteratively outputting shrinkage ratios for hyperparameters such as kernel size or padding. This organization of the action space, together with a smart reward design achieves impressive compression results, given that this approach automates tedious architecture selection. The reward design favors low compression/high accuracy over high compression/low performance while the reward still monotonically increases with both compression and accuracy. As a bonus, the authors also demonstrate how to include hard constraints such as parameter count limitations into the reward model and show that policies trained on small teachers generalize to larger teacher models.
Review:
The manuscript describes the proposed algorithm in great detail and the description is easy to follow. The experimental analysis of the approach is very convincing and confirms the author’s claims.
Using the teacher network as starting point for the architecture search is a good choice, as initialization strategies are a critical component in knowledge distillation. I am looking forward to seeing work on the research goals outlined in the Future Directions section.
A few questions/comments:
1) I understand that L_{1,2} in Algorithm 1 correspond to the number of layers in the network, but what do N_{1,2} correspond to? Are these multiple rollouts of the policies? If so, shouldn’t the parameter update theta_{{shrink,remove},i} be outside the loop over N and apply the average over rollouts according to Equation (2)? I think I might have missed something here.
2) Minor: some of the citations are a bit awkward, e.g. on page 7: “algorithm from Williams Williams (1992). I would use the \citet command from natbib for such citations and \citep for parenthesized citations, e.g. “... incorporate dark knowledge (Hinton et al., 2015)” or “The MNIST (LeCun et al., 1998) dataset...”
3) In Section 4.6 (the transfer learning experiment), it would be interesting to compare the performance measures for different numbers of policy update iterations.
4) Appendix: Section 8 states “Below are the results”, but the figure landed on the next page. I would either try to force the figures to be output at that position (not in or after Section 9) or write "Figures X-Y show the results". Also in Section 11, Figure 13 should be referenced with the \ref command
5) Just to get a rough idea of training time: Could you share how long some of the experiments took with the setup you described (using 4 TitanX GPUs)?
6) Did you use data augmentation for both teacher and student models in the CIFAR10/100 and Caltech256 experiments?
7) What is the threshold you used to decide if the size of the FC layer input yields a degenerate solution?
Overall, this manuscript is a submission of exceptional quality and if minor details of the experimental setup are added to the manuscript, I would consider giving it the full score.


Conventional deep learning classifiers are static in the sense that they are trained on a predefined set of classes and learning to classify a novel class typically requires re-training. In this work, we address the problem of Low-Shot network-expansion learning. We introduce a learning framework which enables expanding a pre-trained (base) deep network to classify novel classes when the number of examples for the novel classes is particularly small. We present a simple yet powerful distillation method where the base network is augmented with additional weights to classify the novel classes, while keeping the weights of the base network unchanged. We term this learning hard distillation, since we preserve the response of the network on the old classes to be equal in both the base and the expanded network. We show that since only a small number of weights needs to be trained, the hard distillation excels for low-shot training scenarios. Furthermore, hard distillation avoids detriment to classification performance on the base classes. Finally, we show that low-shot network expansion can be done with a very small memory footprint by using a compact generative model of the base classes training data with only a negligible degradation relative to learning with the full training set.

The paper proposes a method for adapting a pre-trained network, trained on a fixed number of
classes, to incorporate novel classes for doing classification, especially when the novel classes
only have a few training examples available. They propose to do a `hard' distillation, i.e. they
introduce new nodes and parameters to the network to add the new classes, but only fine-tune the new
networks without modifying the original parameters. This ensures that, in the new expanded and
fine-tuned network, the class confusions will only be between the old and new classes and not
between the old classes, thus avoiding catastrophic forgetting. In addition they use GMMs trained on
the old classes during the fine-tuning process, thus avoiding saving all the original training data.
They show experiments on public benchmarks with three different scenarios, i.e.  base and novel
classes from different domains, base and novel classes from the same domain and novel classes have
similarities among themselves, and base and novel classes from the same domain and each novel class
has similarities with at least one of the base class.

- The paper is generally well written and it is clear what is being done
- The idea is simple and novel; to the best of my knowledge it has not been tested before
- The method is compared with Nearest Class Means (NCM) and Prototype-kNN with soft distillation
  (iCARL; where all weights are fine-tuned). The proposed method performs better in low-shot
  settings and comparably when large number of training examples of the novel classes are available
- My main criticism will be the limited dataset size on which the method is validated. The ILSVRC12
  subset contains 5 base and 5 novel classes and the UT-Zappos50K subset also has 10 classes. The
  idea is simple and novel, which is good, but the validation is limited and far from any realistic
  use. Having only O(10) classes is not convincing, especially when the datasets used do have large
  number of classes. I agree that this will not allow or will takes some involved manual effort to
  curate subsets for the settings proposed, but it is necessary for being convincing.


Published as a conference paper at ICLR 2018 LEARNING FROM NOISY SINGLY-LABELED DATA
Supervised learning depends on annotated examples, which are taken to be the ground truth. But these labels often come from noisy crowdsourcing platforms, like Amazon Mechanical Turk. Practitioners typically collect multiple labels per example and aggregate the results to mitigate noise (the classic crowdsourcing problem). Given a fixed annotation budget and unlimited unlabeled data, redundant annotation comes at the expense of fewer labeled examples. This raises two fundamental questions: (1) How can we best learn from noisy workers? (2) How should we allocate our labeling budget to maximize the performance of a classifier? We propose a new algorithm for jointly modeling labels and worker quality from noisy crowd-sourced data. The alternating minimization proceeds in rounds, estimating worker quality from disagreement with the current model and then updating the model by optimizing a loss function that accounts for the current estimate of worker quality. Unlike previous approaches, even with only one annotation per example, our algorithm can estimate worker quality. We establish a generalization error bound for models learned with our algorithm and establish theoretically that it's better to label many examples once (vs less multiply) when worker quality exceeds a threshold. Experiments conducted on both ImageNet (with simulated noisy workers) and MS-COCO (using the real crowdsourced labels) confirm our algorithm's benefits.

This paper focuses on the learning-from-crowds problem when there is only one (or very few) noisy label per item. The main framework is based on the Dawid-Skene model. By jointly update the classifier weights and the confusion matrices of workers, the predictions of the classifier can help on the estimation problem with rare crowdsourced labels. The paper discusses the influence of the label redundancy both theoretically and empirically. Results show that with a fixed budget, it’s better to label many examples once rather than fewer examples multiple times.
The model and algorithm in this paper are simple and straightforward. However, I like the motivation of this paper and the discussion about the relationship between training efficiency and label redundancy. The problem of label aggregation with low redundancy is common in practice but hardly be formally analyzed and discussed. The conclusion that labeling more examples once is better can inspire other researchers to find more efficient ways to improve crowdsourcing.
About the technique details, this paper is clearly written, but some experimental comparisons and claims are not very convincing. Here I list some of my questions:
+About the MBEM algorithm, it’s better to make clear the difference between MBEM and a standard EM. Will it always converge? What’s its objective?
+The setting of Theorem 4.1 seems too simple. Can the results be extended to more general settings, such as when workers are not identical?
+When n = O(m log m), the result that \epslon_1 is constant is counterintuitive, people usually think large redundancy r can bring benefits on estimation, can you explain more on this?
+During CIFAR-10 experiments when r=1, each example only have one label. For the baselines weighted-MV and weighted-EM, they can only be directly trained using the same noisy labels. So can you explain why their performance is slightly different in most settings? Is it due to the randomly chosen procedure of the noisy labels?
+For ImageNet and MS-COCO experiments with a fixed budget, you reduced the training set when increasing the redundancy, which is unfair. The reduction of performance could mainly cause by seeing fewer raw images, but not the labels. It’s better to train some semi-supervised model to make the settings more comparable.


A BAYESIAN PERSPECTIVE ON GENERALIZATION AND STOCHASTIC GRADIENT DESCENT
We consider two questions at the heart of machine learning; how can we predict if a minimum will generalize to the test set, and why does stochastic gradient descent find minima that generalize well? Our work responds to Zhang et al. (2016), who showed deep neural networks can easily memorize randomly labeled training data, despite generalizing well on real labels of the same inputs. We show that the same phenomenon occurs in small linear models. These observations are explained by the Bayesian evidence, which penalizes sharp minima but is invariant to model parameterization. We also demonstrate that, when one holds the learning rate fixed, there is an optimum batch size which maximizes the test set accuracy. We propose that the noise introduced by small mini-batches drives the parameters towards minima whose evidence is large. Interpreting stochastic gradient descent as a stochastic differential equation, we identify the "noise scale" g = (
where is the learning rate, N the training set size and B the batch size. Consequently the optimum batch size is proportional to both the learning rate and the size of the training set, B opt ∝ N . We verify these predictions empirically.

Summary:
This paper presents a very interesting perspective on why deep neural networks may generalize well, in spite of their high capacity (Zhang et al, 2017). It does so from the perspective of "Bayesian model comparison", where two models are compared based on their "marginal likelihood" (aka, their "evidence" --- the expected probability of the training data under the model, when parameters are drawn from the prior).  It first shows that a simple weakly regularized (linear) logistic regression model over 200 dimensional data can perfectly memorize a random training set with 200 points, while also generalizing well when the class labels are not random (eg, when a simple linear model explains the class labels); this provides a much simpler example of a model generalizing well in spite of high capacity, relative to the experiments presented by Zhang et al (2017). It shows that in this very simple setting, the "evidence" of a model correlates well with the test accuracy, and thus could explain this phenomena (evidence is low for model trained on random data, but high for model trained on real data).
The paper goes on to show that if the evidence is approximated using a second order Taylor expansion of the cost function around a minimia $w_0$, then the evidence is controlled by the cost at the minimum, and by the logarithm of the ratio of the curvature at the minimum compared to the regularization constant (eg, standard deviation of gaussian prior).  Thus, Bayesian evidence prefers minima that are both deep and broad.  This provides a way of comparing models in a way which is independent of the model parametrization (unfortunately, however, computing the evidence is intractable for large networks). The paper then discusses how SGD can be seen as an algorithmic way of finding minima with large "evidence" --- the "noise" in the gradient estimation helps the model avoid "sharp" minima, while the gradient helps the model find "deep" minima.  The paper shows that SGD can be understood using stochastic differential equations, where the noise scale is approximately aN/((1-m)B) (a = learning rate, N = size of training set, B = batch size, m = momentum).  It argues that because there should be an optimal noise scale (which maximizes test performance), the batch size should be taken proportional to the learning rate, as well as the training set size, and proportional to 1/(1-m).  These scaling rules are confirmed experimentally (DNN trained on MNIST).  Thus, this Bayesian perspective can also help explain the observation that models trained with smaller batch sizes (noisier gradient estimates) often generalize better than those with larger batch sizes (Kesker et al, 2016). These scaling rules provide guidance on how to increase the batch size, which is desirable for increasing the parralelism of SGD training.
Review:
Quality: The quality of the work is high.  Experiments and analysis are both presented clearly.
Clarity: The paper is relatively clear, though some of the connections between the different parts of the paper felt unclear to me:
1) It would be nice if the paper were to explain, from a theoretical perspective, why large evidence should correspond to better generalization, or provide an overview of the work which has shown this (eg, Rissanen, 1983).
2) Could margin-based generalization bounds explain the superior generalization performance of the linear model trained on random vs. non-random data?  It seems to me that the model trained on meaningful data should have a larger margin.
3) The connection between the work on Bayesian evidence, and the work on SGD, felt very informal. The link seems to be purely intuitive (SGD should converge to minima with high evidence, because its updates are noisy).  Can this be formalized?  There is a footnote on page 7 regarding Bayesian posterior sampling -- I think this should be brought into the body of the paper, and explained in more detail.
4) The paper does not give any background on stochastic differential equations, and why there should be an optimal noise scale 'g', which remains constant during the stochastic process, for converging to a minima with high evidene.  Are there any theoretical results which can be leveraged from the stochastic processes literature? For example, are there results which prove anything regarding the convergence of a stochastic process under different amounts of noise?
5) It was unclear to me why momentum was used in the MNIST experiments.  This seems to complicate the experimental setting.  Does the generalization gap not appear when no momentum is used?  Also, why is the same learning rate used for both small and large batch training for Figures 3 and 4?  If the learning rate were optimized together with batch size (eg, keeping aN/B constant), would the generalization gap still appear?  Figure 5a seems to suggest that it would not appear (peaks appear to all have the same test accuracy).
6) It was unclear to me whether the analysis of SGD as a stochastic differential equation with noise scale aN/((1-m)B) was a contribution of this paper.  It would be good if it were made clearer which part of the mathematical analysis in sections 2 and 5 are original.
7) Some small feedback: The notation $< x_i > = 0$ and $< x_i^2 > = 1$ is not explained.  Is each feature being normalized to be zero mean, unit variance, or is each training example being normalized?
Originality: The works seems to be relatively original combination of ideas from Bayesian evidence, to deep neural network research.  However, I am not familiar enough with the literature on Bayesian evidence, or the literature on sharp/broad minima, and their generalization properties, to be able to confidently say how original this work is.
Significance: I believe that this work is quite significant in two different ways:
1) "Bayesian evidence" provides a nice way of understanding why neural nets might generalize well, which could lead to further theoretical contributions.
2) The scaling rules described in section 5 could help practitioners use much larger batch sizes during training, by simultaneously increasing the learning rate, the training set size, and/or the momentum parameter.  This could help parallelize neural network training considerably.
Some things which could limit the significance of the work:
1) The paper does not provide a way of measuring the (approximate) evidence of a model.  It simply says it is prohibitively expensive to compute for large models.  Can the "Gaussian approximation" to the evidence (equation 10) be approximated efficiently for large neural networks?
2) The paper does not prove that SGD converges to models of high evidence, or formally relate the noise scale 'g' to the quality of the converged model, or relate the evidence of the model to its generalization performance.
Overall, I feel the strengths of the paper outweight its weaknesses.  I think that the paper would be made stronger and clearer if the questions I raised above are addressed prior to publication.


PARALLELIZING LINEAR RECURRENT NEURAL NETS OVER SEQUENCE LENGTH
Recurrent neural networks (RNNs) are widely used to model sequential data but their non-linear dependencies between sequence elements prevent parallelizing training over sequence length. We show the training of RNNs with only linear sequential dependencies can be parallelized over the sequence length using the parallel scan algorithm, leading to rapid training on long sequences even with small minibatch size. We develop a parallel linear recurrence CUDA kernel and show that it can be applied to immediately speed up training and inference of several state of the art RNN architectures by up to 9x. We abstract recent work on linear RNNs into a new framework of linear surrogate RNNs and develop a linear surrogate model for the long short-term memory unit, the GILR-LSTM, that utilizes parallel linear recurrence. We extend sequence learning to new extremely long sequence regimes that were previously out of reach by successfully training a GILR-LSTM on a synthetic sequence classification task with a one million timestep dependency.

# Summary and Assessment
The paper addresses an important issue–that of making learning of recurrent networks tractable for sequence lengths well beyond 1’000s of time steps. A key problem here is that processing such sequences with ordinary RNNs requires a reduce operation, where the output of the net at time step t depends on the outputs of *all* its predecessor.
The authors now make a crucial observation, namely that a certain class of RNNs allows evaluation in a non-linear fashion through a so-called SCAN operator. Here, if certain conditions are satisfied, the calculation of the output   can be parallelised massively.
In the following, the authors explore the landscape of RNNs satisfying the necessary conditions. The performance is investigated in terms of wall clock time. Further, experimental results of problems with previously untacked sequence lengths are reported.
The paper is certainly relevant, as it can pave the way towards the application of recurrent architectures to problems that have extremely long term dependencies.
To me, the execution seems sound. The experiments back up the claim.
## Minor
- I challenge the claim that thousands and millions of time steps are a common issue in “robotics, remote sensing, control systems, speech recognition, medicine and finance”, as claimed in the first paragraph of the introduction. IMHO, most problems in these domains get away with a few hundred time steps; nevertheless, I’d appreciate a few examples where this is a case to better justify the method.


We achieve bias-variance decomposition for Boltzmann machines using an information geometric formulation. Our decomposition leads to an interesting phenomenon that the variance does not necessarily increase when more parameters are included in Boltzmann machines, while the bias always decreases. Our result gives a theoretical evidence of the generalization ability of deep learning architectures because it provides the possibility of increasing the representation power with avoiding the variance inflation.

Summary of the paper:
The paper derives a lower bound on the expected  squared KL-divergence between a true distribution and the sample based maximum likelihood estimate (MLE) of that distribution modelled by an Boltzmann machine (BM) based on methods from information geometry. This  KL-divergence is first split into the squared KL-divergence between the true distribution and MLE of that distribution,  and the expected squared KL-divergence between the MLE of the true distribution and the sample based MLE (in a similar spirit to splitting the excess error into approximation and estimation error in statistical learning theory). The letter is than lower bounded (leading to a lower bound on the overall KL-divergence) by a term  which does not necessarily increase if the number of model parameters is increased.
Pros:
- Using insights from information geometry  opens up a very interesting and (to my knowledge) new approach for analysing the generalisation ability of ML models.
- I am not an expert on information geometry and I did not find the time to follow all the steps of the proof in detail, but the analysis seems to be correct.
Cons:
- The fact that the lower bound does not necessary increase with a growing number of parameters does not guarantee that the same holds true for the KL-divergence (in this sense an upper bound would be more informative). Therefore, it is not clear how much of insights the theoretical analysis gives for practitioners (it could be nice to analyse the tightness of the bound for toy models).
- Another drawback reading the practical impact is, that the theorem bounds the expected  squared KL-divergence between a true distribution and the sample based MLE, while training minimises the divergence between the empirical distribution and the model distribution ( i.e. the sample based MLE in the optimal case),  and the theorem does not show the dependency on the letter.
I found some parts difficulty to understand and clarity could be improved  e.g. by
- explaining why minimising KL(\hat P, P_B) is equivalent to minimising the KL-divergence between the empirical distribution and the Gibbs distribution \Phi.
- explaining in which sense the formula on page 4 is equivalent to “the learning equation of Boltzmann machines”.
- explaining what is the MLE of the true distribution (I assume the closest distribution in the set of distributions that can be modelled by the BM).
Minor comments:
- page 1: and DBMs….(Hinton et al., 2006) : The paper describes deep belief networks (DBNs) not DBMs
- \theta is used to describe the function in eq. (2) as well as the BM parameters in Section 2.2
- page 5: “nodes H is” -> “nodes H are”
REVISION:
Thanks to the reviewers for replying to my comments and making the changes. I think they improved the paper. On the other hand the other reviewers raised valid questions, that led to my decision to not change the overall rating of the paper.


Recent work has shown that performing inference with fast, very-low-bitwidth (e.g., 1 to 2 bits) representations of values in models can yield surprisingly accurate results. However, although 2-bit approximated networks have been shown to be quite accurate, 1 bit approximations, which are twice as fast, have restrictively low accuracy. We propose a method to train models whose weights are a mixture of bitwidths, that allows us to more finely tune the accuracy/speed trade-off. We present the "middle-out" criterion for determining the bitwidth for each value, and show how to integrate it into training models with a desired mixture of bitwidths. We evaluate several architectures and binarization techniques on the ImageNet dataset. We show that our heterogeneous bitwidth approximation achieves superlinear scaling of accuracy with bitwidth. Using an average of only 1.4 bits, we are able to outperform state-of-the-art 2-bit architectures.

This paper suggests a method for varying the degree of quantization in a neural network during the forward propagation phase.
Though this is an important direction to investigate, there are several issues:
1. Comparison with previous results is misleading:
a.	1-bit weights and floating point activations: Rastegari et al. got 56.8% accuracy on Alexnet, which is better than this paper 1.4bit result of 55.2%.
b.	Hubara et al. got 51% results on 1-bit weights and 2-bit activations included also quantization first and last layer, in contrast to this paper. Therefore, it is not clear if there is a significant benefit in the proposed method which achieves 51.5% when decreasing the activation precision to 1.4bit.
Therefore, it is not clear that the proposed methods improve over previous approaches.
2. It is not clear to me: in which dimension of the tensors are we saving the scale factor? If it is per feature map, or neuron, this eliminates the main benefits of quantization: doing efficient binarized operations when doing Weight*activation during the forward pass?
3. The review of the literature is inaccurate. For example, it is not true that Courbariaux et al. (2016) “further improved accuracy on small datasets”: the main novelty there was binarizing the activations (which typically decreased the accuracy). Also, it is not clear if the scale factors introduced by XNOR-Net indeed allowed "a significant improvement over previous work" in ImageNet (e.g., see DoReFA and Hubara et al. who got similar results using binarized weigths and activations on ImageNet without scale factors).  Lastly, the statement “Typical approaches include linearly placing the quantization points” is inaccurate: it was observed that logarithmic quantization works better in various cases. For example, see Miyashita, Lee and Murmann 2016, and Hubara et al.
%%% After Author's Clarification %%%
This paper results seem more positive now, and I have therefore have increased my score, assuming the authors will revise the paper accordingly.


SMASH: ONE-SHOT MODEL ARCHITECTURE SEARCH THROUGH HYPERNETWORKS
Designing architectures for deep neural networks requires expert knowledge and substantial computation time. We propose a technique to accelerate architecture selection by learning an auxiliary HyperNet that generates the weights of a main model conditioned on that model's architecture. By comparing the relative validation performance of networks with HyperNet-generated weights, we can effectively search over a wide range of architectures at the cost of a single training run. To facilitate this search, we develop a flexible mechanism based on memory read-writes that allows us to define a wide range of network connectivity patterns, with ResNet, DenseNet, and FractalNet blocks as special cases. We validate our method (SMASH) on CIFAR-10 and CIFAR-100, STL-10, ModelNet10, and Imagenet32x32, achieving competitive performance with similarly-sized handdesigned networks.

Summary of paper - This paper presents SMASH (or the one-Shot Model Architecture Search through Hypernetworks) which has two training phases (one to quickly train a random sample of network architectures and one to train the best architecture from the first stage). The paper presents a number of interesting experiments and discussions about those experiments, but offers more exciting ideas about training neural nets than experimental successes.
Review - The paper is very well written with clear examples and an excellent contextualization of the work among current work in the field. The introduction and related work are excellently written providing both context for the paper and a preview of the rest of the paper. The clear writing make the paper easy to read, which also makes clear the various weaknesses and pitfalls of SMASH.
The SMASH framework appears to provide more interesting contributions to the theory of training Neural Nets than the application of said training. While in some experiments SMASH offers excellent results, in others the results are lackluster (which the authors admit, offering possible explanations).
It is a shame that the authors chose to push their section on future work to the appendices. The glimmers of future research directions (such as the end of the last paragraph in section 4.2) were some of the most intellectually exciting parts of the paper. This choice may be a reflection of preferring to highlight the experimental results over possible contributions to theory of neural nets.
Pros -
* Strong related work section that contextualizes this paper among current work
* Very interesting idea to more efficiently find and train best architectures
* Excellent and thought provoking discussions of middle steps and mediocre results on some experiments (i.e. last paragraph of section 4.1, and last paragraph of section 4.2)
* Publicly available code
Cons -
* Some very strong experimental results contrasted with some mediocre results
* The balance of the paper seems off, using more text on experiments than the contributions to theory.
* (Minor) - The citation style is inconsistent in places.
=-=-=-= Response to the authors
I thank the authors for their thoughtful responses and for the new draft of their paper. The new draft laid plain the contribution of the memory bank which I had missed in the first version. As expected, the addition of the future work section added further intellectual excitement to the paper.
The expansion of section 4.1 addressed and resolved my concerns about the balance of the paper by effortless intertwining theory and application. I do have one question from this section -  In table 1, the authors report p-values but fail to include them in their interpretation; what is purpose of including these p-values, especially noting that only one falls under the typical threshold for significance?


Generative modeling of high dimensional data like images is a notoriously difficult and ill-defined problem. In particular, how to evaluate a learned generative model is unclear. In this paper, we argue that adversarial learning, pioneered with generative adversarial networks (GANs), provides an interesting framework to implicitly define more meaningful task losses for unsupervised tasks, such as for generating "visually realistic" images. By relating GANs and structured prediction under the framework of statistical decision theory, we put into light links between recent advances in structured prediction theory and the choice of the divergence in GANs. We argue that the insights about the notions of "hard" and "easy" to learn losses can be analogously extended to adversarial divergences. We also discuss the attractive properties of parametric adversarial divergences for generative modeling, and perform experiments to show the importance of choosing a divergence that reflects the final task.

This paper introduces a family of "parametric adversarial divergences" and argue that they have advantages over other divergences in generative modelling, specially for structured outputs.
There's clear value in having good inductive biases (e.g. expressed in the form of the discriminator architecture) when defining divergences for practical applications. However, I think that the paper would be much more valuable if its focus shifted from presenting a new notion of divergence to deep-diving into the effect of inductive biases and presenting more specific results (theoretical and / or empirical) in structured prediction or other problems.  In its current form the paper doesn't seem particularly strong for either the divergence or GAN literatures. Some reasons below:
* There are no specific results on properties of the divergences, or axioms that justify them. I think that presenting a very all-encompassing formulation without a strong foundation does not add value.
* There's abundant literature on f-divergences which show that there's a 1-1 relationship between divergences and optimal (Bayes) risks of classification problems (e.g. Reid at al. Information, Divergence and Risk for Binary Experiments in JMLR and Garcia-Garcia et al. Divergences and Risks for Multiclass Experiments in COLT).  This disproves the point that the authors make that it's not possible to encode information about the final task in the divergence. If the loss for the task is proper, then it's well known how to construct a divergence which coincides with the optimal risk.
* The divergences presented in this work are different from the above since the risk is minimised over a parametric class instead of over the whole set of integrable functions. However, practical estimators of f-divergences also reduce the optimization space (e.g. unit ball in a RKHS as in Nguyen et al.  Estimating Divergence Functionals and the
Likelihood Ratio by Convex Risk Minimization or Ruderman et al. Tighter Variational Representations of f-Divergences via Restriction to Probability Measures). So, given the lack of strong foundation for the formulation, "parametric adversarial divergences" feel more like estimators of other divergences than a relevant new family.
* There are many estimators for f-divergences (like the ones cited above and many others based e.g. on nearest-neighbors) that are sample-based and thus correspond to the "implicit" case that the authors discuss. They don't necessarily need to use the dual form. So table 1 and the first part of Section 3.1 are not accurate.
* The experiments are few and too specific, specially given that the paper presents a very general framework. The first experiment just shows that Wasserstein GANs don't perform well in an specific dataset and use that to validate a point about those GANs not being good for high dimensions due to their sample complexity. That feels like confirmation bias and also does not really say anything about the parametric adversarial GANs, which are the focus of the paper.
In summary, I like the authors idea to explore the restriction of the function class of dual representations to produce useful-in-practice divergences, but the paper feels a bit middle of the road. The theory is not strong and the experiments don't necessary support the intuitive claims made in the paper.