CNNpack: Packing Convolutional Neural Networks in the Frequency Domain
Deep convolutional neural networks (CNNs) are successfully used in a number of applications. However, their storage and computational requirements have largely prevented their widespread use on mobile devices. Here we present an effective CNN compression approach in the frequency domain, which focuses not only on smaller weights but on all the weights and their underlying connections. By treating convolutional filters as images, we decompose their representations in the frequency domain as common parts (i.e., cluster centers) shared by other similar filters and their individual private parts (i.e., individual residuals). A large number of low-energy frequency coefficients in both parts can be discarded to produce high compression without significantly compromising accuracy. We relax the computational burden of convolution operations in CNNs by linearly combining the convolution responses of discrete cosine transform (DCT) bases. The compression and speed-up ratios of the proposed algorithm are thoroughly analyzed and evaluated on benchmark image datasets to demonstrate its superiority over state-of-the-art methods.

This paper presents a framework of compressing and speeding up convolutional neural networks. The convolutional kernels are transformed into frequency domain with Discrete Cosine Transform. k-means clustering, l1 shrinking, quantization and Huffman coding is performed sequentially to reduce the consumption of memory and computational complexity. Excellent experimental results are achieved on several state of the art CNN models.
1. The experimental results are exceptionally good, significantly better than previous methods. The authors evaluated their method on Alexnet, VGG and Googlenet, which I think are sufficiently convincing.  2. Concerns about novelty: Comparing to [1], the main added part is the DCT transformation.  While speeding up CNN with frequency domain has been proposed in [2].   3. The speedup convolution section seems confusing to me.      3.1 I believe the DCT mentioned in this paper is 2-dimensional DCT, as is defined in equation (2). The complexity of 2D DCT of a d x d patch should not be O(dlog(d)). I believe O(dlog(d)) is the complexity of 1D DCT with d elements.      3.2 Line 166 to 168 seems to be misleading. The author says that convolving with the d x d DCT bases is equivalent to performing DCT direct to the H x W input data. However, it should be performing d x d DCT at each location of the H x W input data. These are two different operations.     3.3 When calculating complexity, the authors never mentioned the number of input channels. Is the number of input channels supposed to be one?    4. I suggest the authors to compare their method with [3]. DCT is a type of decomposition with fixed basis, while in [3], sparse decomposition was adopted, and the basis is optimized while fine-tuning.  5. I'm interested in the values of eta and delta for the models evaluated in the experiments. eta and delta provides the level of sparsity for the k-means centers and residuals. This can be compared with the sparsity obtained in [3].  6. The run time of proposed method is given in Table 2 of the manuscript. I'm very curious about how the authors implemented their method with GPU. According to [3], performing sparse matrix multiplication heavily suffers from the limited memory bandwidth, and GPU is well known to be very hard to optimize. Considering the proposed method is relatively complicated, I'm wondering how the authors parallelize it efficiently with CUDA and achieve runtime speedup that is similar to theoretical speedup. I suggest the authors compare their efficiency with the cudnn implementation, which represents the state-of-the-art implementation of CNN models.  [1] Han, Song, Huizi Mao, and William J. Dally. "Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding." CoRR, abs/1510.00149 2 (2015). [2] Mathieu, Michael, Mikael Henaff, and Yann LeCun. "Fast training of convolutional networks through FFTs." arXiv preprint arXiv:1312.5851 (2013). [3] Liu, Baoyuan, et al. "Sparse convolutional neural networks." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015.


Deep Learning Models of the Retinal Response to Natural Scenes
A central challenge in sensory neuroscience is to understand neural computations and circuit mechanisms that underlie the encoding of ethologically relevant, natural stimuli. In multilayered neural circuits, nonlinear processes such as synaptic transmission and spiking dynamics present a significant obstacle to the creation of accurate computational models of responses to natural stimuli. Here we demonstrate that deep convolutional neural networks (CNNs) capture retinal responses to natural scenes nearly to within the variability of a cell's response, and are markedly more accurate than linear-nonlinear (LN) models and Generalized Linear Models (GLMs). Moreover, we find two additional surprising properties of CNNs: they are less susceptible to overfitting than their LN counterparts when trained on small amounts of data, and generalize better when tested on stimuli drawn from a different distribution (e.g. between natural scenes and white noise). An examination of the learned CNNs reveals several properties. First, a richer set of feature maps is necessary for predicting the responses to natural scenes compared to white noise. Second, temporally precise responses to slowly varying inputs originate from feedforward inhibition, similar to known retinal mechanisms. Third, the injection of latent noise sources in intermediate layers enables our model to capture the sub-Poisson spiking variability observed in retinal ganglion cells. Fourth, augmenting our CNNs with recurrent lateral connections enables them to capture contrast adaptation as an emergent property of accurately describing retinal responses to natural scenes. These methods can be readily generalized to other sensory modalities and stimulus ensembles. Overall, this work demonstrates that CNNs not only accurately capture sensory circuit responses to natural scenes, but also can yield information about the circuit's internal structure and function.

This interesting study models the response of ganglion cells in the retina of a salamander. To the best of my knowledge this is the first time a convolutional neural network is applied to this fairly well studied modeling problem. The authors report better than state of the art performance. Furthermore, and perhaps most interesting in my humble opinion, they report that these models generalize relatively well between artificial white noise stimuli and natural stimuli. This has been a glaring weakness of previous models. Thus I find these results novel and exciting.
Modeling studies of neural responses are usually measured on two scales: a. Their contribution to our understanding of the neural physiology, architecture or any other biological aspect. b. Model accuracy, where the aim is to provide a model which is better than the state of the art. To the best of my understanding, this study mostly focuses on the latter, i.e. provide a better than state of the art model. If I am misunderstanding, then it would probably be important to stress the biological insights gained from the study. Yet if indeed modeling accuracy is the focus, it's important to provide a fair comparison to the state of the art, and I see a few caveats in that regard: 1. The authors mention the GLM model of Pillow et al. which is pretty much state of the art, but a central point in that paper was that coupling filters between neurons are very important for the accuracy of the model. These coupling filters are omitted here which makes the comparison slightly unfair. I would strongly suggest comparing to a GLM with coupling filters. Furthermore, I suggest presenting data (like correlation coefficients) from previous studies to make sure the comparison is fair and in line with previous literature. 2. The authors note that the LN model needed regularization, but then they apply regularization (in the form of a cropped stimulus) to both LN models and GLMs. To the best of my recollection the GLM presented by pillow et al. did not crop the image but used L1 regularization for the filters and a low rank approximation to the spatial filter. To make the comparison as fair as possible I think it is important to try to reproduce the main features of previous models.  Minor notes: 1. Please define the dashed lines in fig. 2A-B and 4B. 2. Why is the training correlation increasing with the amount of training data for the cutout LN model (fig. 4A)? 3. I think figure 6C is a bit awkward, it implies negative rates, which is not the case, I would suggest using a second y-axis or another visualization which is more physically accurate. 4. Please clarify how the model in fig. 7 was trained. Was it on full field flicker stimulus changing contrast with a fixed cycle? If the duration of the cycle changes (shortens, since as the authors mention the model cannot handle longer time scales), will the time scale of adaptation shorten as reported in e.g Smirnakis et al. Nature 1997.


Convergence guarantees for kernel-based quadrature rules in misspecified settings
Kernel-based quadrature rules are becoming important in machine learning and statistics, as they achieve super-√ n convergence rates in numerical integration, and thus provide alternatives to Monte Carlo integration in challenging settings where integrands are expensive to evaluate or where integrands are high dimensional. These rules are based on the assumption that the integrand has a certain degree of smoothness, which is expressed as that the integrand belongs to a certain reproducing kernel Hilbert space (RKHS). However, this assumption can be violated in practice (e.g., when the integrand is a black box function), and no general theory has been established for the convergence of kernel quadratures in such misspecified settings. Our contribution is in proving that kernel quadratures can be consistent even when the integrand does not belong to the assumed RKHS, i.e., when the integrand is less smooth than assumed. Specifically, we derive convergence rates that depend on the (unknown) lesser smoothness of the integrand, where the degree of smoothness is expressed via powers of RKHSs or via Sobolev spaces.

%%% UPDATE %%% Thank you for your concise response. I remain confident that this paper would make a solid contribution to NIPS this year. %%% END %%%   This paper extends results on kernel quadrature to the misspecified setting, where the integrand is not smooth enough to belong to the RKHS H characterised by the kernel. The main result of the paper is that kernel quadrature is adaptive to unknown smoothness; this is provided that the integrand belongs to a RKHS on the Hilbert scale between H and L2.
I would like to congratulate the authors on an excellent paper. This is an important contribution to the literature on kernel quadrature and established useful theoretical guidance to practitioners - namely, to err on the side of using kernels that are `too smooth' and allow automatic adaptation to occur. Given the importance of numerical integration, I expect this work to have substantial impact.  I have some very minor comments.  The literature tends to refer to "kernel quadrature" when the weights w_i are those that minimise the worst-case error for fixed locations X_i, i.e. the weights w_i that are used in Bayesian quadrature. This agrees with your definition on page 3, line 105. However, defined in this way, quasi Monte Carlo (QMC) is *not* kernel quadrature (the weights are uniform weights). I do take the authors' point that QMC can be studied in RKHS, but I would clarify the wording on page 1, line 26 to emphasise that QMC is not itself a kernel quadrature method.  This work is close in spirit to ref [17] in the paper. In [17] it was argued that a QMC lattice rule for functions that are rougher than the integrand could be compensated for by building control variates with a kernel that was smoother than the integrand. The argument of adapting to unknown smoothness was also made in [17]. I believe the present paper implies that an alternative to [17] is to simply use a QMC lattice rule that is for smoother functions. This work is perhaps a nicer solution compared to [17].  On page 3, line 102, the phrase "it suffices" is a bit misleading. You are *choosing* to pursue a minimax strategy, but until this point the goal was not stated in terms this explicit.  In the supplement on page 1, line 13, I don't think Smale and Zhou (2007) is the original reference for this result (they cite another work).


Efficient High-Order Interaction-Aware Feature Selection Based on Conditional Mutual Information
This study introduces a novel feature selection approach CMICOT, which is a further evolution of filter methods with sequential forward selection (SFS) whose scoring functions are based on conditional mutual information (MI). We state and study a novel saddle point (max-min) optimization problem to build a scoring function that is able to identify joint interactions between several features. This method fills the gap of MI-based SFS techniques with high-order dependencies. In this high-dimensional case, the estimation of MI has prohibitively high sample complexity. We mitigate this cost using a greedy approximation and binary representatives what makes our technique able to be effectively used. The superiority of our approach is demonstrated by comparison with recently proposed interactionaware filters and several interaction-agnostic state-of-the-art ones on ten publicly available benchmark datasets.

This paper presents a new feature selection algorithm, named CMICOT, that considers high-order dependences among features (up to order t, with t > 3), and uses Mutual Information (MI) to measure such dependences. Additionally, in order to alleviate the lack of data and the increase on computational cost to properly estimate the mutual information, the CMICOT algorithm is a greedy approach that uses binary representatives of each feature.  The proposed algorithm is tested against some state-of-the-art feature selection algorithms, and interaction-aware SFS-based filters. The presented results on 10 public datasets show that overall the CMICOT algorithm outperforms the other feature selection algorithm in terms of classification accuracy when kNN or AdaBoost are used as classifiers.
The problem of designing a feature selection algorithm capable of efficiently deal with high-order interaction among features is an interesting and open problem in the feature selection area. That is why this paper is appealing. However, there are several issues regarding the computational cost and the experimental setup that need a clarification in order to consider it for acceptance.  • My main concern is about the justification and analysis of the Binary representatives procedure described in Section 3.3, which I think compromises the acceptance of this paper. • On the one and, the theoretical justification of this procedure is not very convincing. It is said (lines 219-222) that “The described technique has been inspired by the intuition that probably two binary representatives of two different features interact on average better than two binary representatives of one feature”; however, no references or examples are provided to support this idea.  On the other hand, when comparing the computational cost between the algorithm with and without binary representations (lines 215-219), the same values for t and s are considered. This is not a fair comparison as both cases are not taking the same level of information. Additionally, I have serious doubts about the reduction in computational cost of the binary representatives. Then, in Algorithm 2, when searching for complementary features and opposing features, I can see a reduction in the cost of computing the mutual information (we are working with binary variables), but the cardinalities of S^bin U B[f] and S^bin are larger than in the non-binary case, as all the set B[f_best] is added to the set S^bin set in line 21. In short, it seems like there is a trade-off between the computational cost to compute the argmax and the mutual information. • Lines 180-182: If an “optimal” interaction-aware MI-based feature selection method  is wanted and not a low dimensional approximation of G, the proposed method as a computational cost O(i^2), which is the same as RelaxMRMR. What are reasonable values for t and s? Please, clarify this point.  • Please, rephrase/clarify lines 183-191. • I don’t understand why the particular case t=s is described in Algorithm 1, and a generic algorithm is not provided. • Proposition 3. First sentence in Appendix A.4 needs clarification and it is fundamental for the correctness of Proposition 3 (“Proof. The calculation of a joint entropy of m variables over N requires takes O(mN) simple operations. Hence, any MI that involve m variables requires O(mN) simple operations as well.”). • Line 249-250: “but estimation of…. BR technique.” Please clarify this sentence. Isn’t it  a problem-dependent factor? • Experimental setup. There is another critical point here. It is not clear whether the curves presented in Figure 1 are obtained over a validation/test set or over the training set. It is said later on that 10-fold cross-validation is applied to estimate the significance of differences in classification quality, but it is not clear whether this procedure was also applied in Figure 1.This must be clarified. • The proposed method does not provide good results for the NBC classifier; though these results are provided in the appendices, they are not mentioned or even discussed in the main paper. • There is relevant information in the appendices regarding the comparison between different feature selection methods. I think that the average results reported in the main manuscript are not enough. • Experiments. How can it be explained that the strongest competitor is CMIM, and not one of the interaction-aware FS methods? • It would be interesting to empirically compare CMICOT with and without binary representatives in terms of classification accuracy and computational cost. Additionally, it is highly advisable to include some comparisons with the other feature selection methods in terms of computational cost. It is especially critical to make it clear the advantage of the proposed method with respect to other interaction-aware SFS-based filters.  • Minor comments: - Line 27: Should it be SFS-based filters instead of SBS-based filters? - Lines 144-145: clarify the sentence in cursive. - Missing axes labels in Figure 1. - Please, indicate the statistical test used in Table 1.


Using Fast Weights to Attend to the Recent Past
Until recently, research on artificial neural networks was largely restricted to systems with only two types of variable: Neural activities that represent the current or recent input and weights that learn to capture regularities among inputs, outputs and payoffs. There is no good reason for this restriction. Synapses have dynamics at many different time-scales and this suggests that artificial neural networks might benefit from variables that change slower than activities but much faster than the standard weights. These "fast weights" can be used to store temporary memories of the recent past and they provide a neurally plausible way of implementing the type of attention to the past that has recently proved very helpful in sequence-to-sequence models. By using fast weights we can avoid the need to store copies of neural activity patterns.

This paper presents a method for using a rapidly changing secondary matrix of temporary weight values, referred to as "fast weights", in recurrent neural networks to maintain a trace of past inputs and network states. Presented experiments show that incorporating these fast weights in recurrent neural networks improve performance. RNNs with fast weights are shown to outperform other RNN variants with equivalent numbers of recurrent units on associative retrieval of characters from strings and image recognition tasks (MNIST and facial expression).
The presented method for incorporating fast weights into recurrent neural networks is elegant and produces noticeable improvements in performance in the experiments presented. The paper is very well written, for the most part. A small amount of related prior work exists on fast weights; however, it seems that research on this topic never caught on. This paper will hopefully change this; fast weights should be of interest to the machine learning community. Additionally, the developments and results presented in this paper have a biological inspiration and may inspire work in computational neuroscience and cognitive science.  Some feedback, questions and suggestions arose on reading:  The performance improvements shown in experimental results indicate that work in fast weights has significant promise. It would have been interesting to see some analysis or an example of the low-level outcomes of fast weight operation, such as a resulting fast weight matrix (in preference to the description of a second image recognition task, perhaps). An appendix describing implementation details was mentioned in the paper but was not submitted with the paper for review.  The contents of the Conclusion section might be better described as, and converted into, a Discussion section (an additional Conclusion section might not be necessary). Suggestions and comments on the Conclusion(/Discussion) follow:  1) The authors may disagree and disregard this suggestion; however, rather than outright state that the paper contributes to a field of study, my preference is to refer to the specific developments as "contributions" and then describe how these contributions are significant or relevant to the fields of study. For example, in the case of the paper under review, the contributions of the paper are a method for adapting fast weights in recurrent neural networks and the experimental results demonstrating improved RNN performance. The demonstration of improved performance of RNNs is evidence of the significance of the contributions to machine learning. These contributions are also relevant to computational neuroscience and cognitive science as a model and evidence that fast mechanisms of synaptic plasticity may contribute to sequential processing of stimuli and working memory.  2) Line 288 has the statement "Layer normalization makes this kind of attention work better", which is mentioned earlier but not actually shown in the paper, possibly making the statement unsuitable as a conclusion in the paper. A statement reminding the reader of the use of layer normalization might be more suitable.  3) References are made to "sequence-to-sequence RNNs used in machine translation" (line 290) and "[t]he ability of people to recursively apply the very same knowledge and processing apparatus to a whole sentence and to an embedded clause within that sentence has long been used as an argument against neural networks as a model of higher-level cognitive abilities" (line 294-297). Ideally, both of these references would be accompanied by a citation.     A few suggestions for presentation and typographical corrections follow:  1) The black-and-white printability of the paper could be improved by choosing thicker line weights for figures 1, 3, and 5, and by using different line styles or arrow heads to distinguish signals. Figure 5 could benefit from a larger font size for plots.  2) Line 203: remove "their". Page 8 has a number of instances of the opening quotation mark being a closing quotation mark. Line 287: "unis" was probably meant to be "units". References [8] and [25] have initialisms that should be capitalised.


A Credit Assignment Compiler for Joint Prediction
Many machine learning applications involve jointly predicting multiple mutually dependent output variables. Learning to search is a family of methods where the complex decision problem is cast into a sequence of decisions via a search space. Although these methods have shown promise both in theory and in practice, implementing them has been burdensomely awkward. In this paper, we show the search space can be defined by an arbitrary imperative program, turning learning to search into a credit assignment compiler. Altogether with the algorithmic improvements for the compiler, we radically reduce the complexity of programming and the running time. We demonstrate the feasibility of our approach on multiple joint prediction tasks. In all cases, we obtain accuracies as high as alternative approaches, at drastically reduced execution and programming time.

This paper presents a system designed to speed up the implementation and execution of learning-to-search algorithms. Once the user implements their application as a small program built atop two library functions ("Predict" and "Loss"), the system can use that program to both learn the model parameters and generate predictions at test time, essentially by swapping in different versions of the two library primitives. Experiments show that several NLP problems can be solved very quickly and accurately using this system, which the authors call a "credit assignment compiler".
This is an interesting and novel paper. It's sort of a fusion between proposing a general "master algorithm" for a number of existing learn-to-search approaches (Searn, DAgger, etc.) and an engineering exercise in making this master algorithm run quickly. The empirical runtime/accuracy results are compelling, and I suspect the system might be useful to a wide range of people once it is released.  At the same time, I have some reservations about the way the paper is written and structured. Since the authors are emphasizing that their implementation of this general algorithm is easy and fast for practitioners to use, it seems problematic that it is (to me) very difficult to understand how one would actually use it. Partly this is because, for instance, Algorithm 1 (MyRun) uses a bunch of terms that have not been explained at that point (tags, etc.) -- i.e., some reordering might help. But partly it's because some things are never really explained.  For instance, other than reverse-engineering the desired learning procedure using Algorithm 3, how should a user write MyRun? How should we think of Predict and Loss in an intuitive way? And how is this different from writing a search algorithm? The concept of a TDOLR function is defined without much context, but seems to be central. There is a footnote about letting MyRun control the machine stack, but I am not sure what this is supposed to mean for pseudocode. And to be honest, I still don't understand why this is called a credit assignment compiler. I get that search provides credit assignment by measuring the effect of each decision in (relative) isolation, but in what sense is this a compiler?  In general, I found the idea of the paper and the results produced compelling, but wished the middle part was better able to articulate why I should believe that this system is natural or easy to use. Some additional examples, tutorial-style, would go a long way. There are also a large number of typos and grammatical errors that need to be fixed.


Accelerating Stochastic Composition Optimization
Consider the stochastic composition optimization problem where the objective is a composition of two expected-value functions. We propose a new stochastic firstorder method, namely the accelerated stochastic compositional proximal gradient (ASC-PG) method, which updates based on queries to the sampling oracle using two different timescales. The ASC-PG is the first proximal gradient method for the stochastic composition problem that can deal with nonsmooth regularization penalty. We show that the ASC-PG exhibits faster convergence than the best known algorithms, and that it achieves the optimal sample-error complexity in several important special cases. We further demonstrate the application of ASC-PG to reinforcement learning and conduct numerical experiments.

The paper extends the accelerated SCGD algorithm in [Wang et.al, 2016] with proximal setting to handle nonsmooth regularization, and present refined convergence results in some cases. Some experiment is conducted on reinforcement learning on small synthetic dataset.
First of all, the novelty and originality is a big concern.   - Most elements are inherited from an early journal paper [Wang et.al, 2016], including the motivation example, the accelerated algorithm itself, results for strongly convex case, and application to dynamic programming.    - The only novelty of the paper is to extend ASCG to proximal setting, which however, is quite trivial. Not to mention the fact that,  the main Theorem 1 assumes no nonsmooth penalty term. So literally, this paper has not really done what they claimed - address nonsmooth regularization penalty without deteriorating the convergence rate.    -The "optimally" strongly convexity condition is not substantially different from strongly convexity. Theorem 2 is more or less just a repetition of Theorem 7 in [Wang et.al, 2016], except with a slightly more general choice of learning rate.   Overall, I don't feel the paper has a substantial contribution.    Secondly, several statements are confusing and could be misleading.   1. The authors claim that the inner and outer random variables w, v can be dependent, but assumes unbiased stochastic oracles given in Assumption 1.1. This seems to be a contradiction, if (w,v) are dependent, then Assumption 1.1 might not be possible. So it feels like the algorithm is limited to independent (w,v). Due to this same reason, the proposed algorithm might not handle the motivating example,  risk-averse learning, since the inner and outer random variables are the same.  2. Somehow the two motivating examples cannot be directly or explicitly reformulated to the form (1). I would suggest the authors to provide some details here on what are the exact functions f, g.  Also, the reformulation (3) does not look quite right to me.   3. In theorem 1, the convergence criterion (taking average of gradient norms) is not common nor expressive enough to imply the rate. It would be more useful to see directly how the gradient of the last iterate converge to zero. Since the criterion used here is different from [Wang et.al, 2016], it is not very appropriate to compare the rates or claim any kind of improvement.    Thirdly, the applicability of the proposed algorithm in reinforcement learning seems quite limited.    - In order to  translate the Bellman residual minimization problem into (1), an extra O(S) inner expectation terms needed to be introduced, which does not look very elegant. Also, this implies that the proposed algorithm can only handle finite state MDP with reasonably small number of states. As shown in their experiments, the largest instance  has only 100 states.   - For the interest of machine learning community, results on real and large-scale datasets should be provided.   Lastly, here are some minor issues.   1. The authors claim that the objective function (3) is optimally strongly convex and later use it in experiments,  so either proof or references should be provided.  2. In Experiment 1 and 2, there is no nonsmooth penalty, so the proposed ASC-PG algorithm is almost nothing different from the ASC-GD algorithm except with a smarter choice of learning rate. The authors should clarify this when making the comparison. In Experiment 3, it also makes sense to compare to the proximal extension of GTD2-MP algorithm.   3. The following reference, also provides a O(1/t) convergence rate for stochastic composition problems when inner function is linear. The authors might want to provides some discussion on the algorithmic differences.  Dai, Bo, et al. "Scalable kernel methods via doubly stochastic gradients." Advances in Neural Information Processing Systems. 2014.  4. Line 139: function F(x) is not defined until in line 144.    5. Line 53:  nonlinearity in distribution of v. Shouldn't it be w?  6. Line 228: ||w_k-w^*||=O(1/k) missing square?


Graphons, mergeons, and so on!
In this work we develop a theory of hierarchical clustering for graphs. Our modeling assumption is that graphs are sampled from a graphon, which is a powerful and general model for generating graphs and analyzing large networks. Graphons are a far richer class of graph models than stochastic blockmodels, the primary setting for recent progress in the statistical theory of graph clustering. We define what it means for an algorithm to produce the "correct" clustering, give sufficient conditions in which a method is statistically consistent, and provide an explicit algorithm satisfying these properties.

Hierarchical clustering is commonly applied to two types of objects: 1. sets of points 2. graphs (in which case it is usually called hierarchical graph partitioning)  What can be said about the statistical properties of hierarchical clustering? In case (1), we can look at the underlying density from which the points are sampled, define a suitable (infinite) "cluster tree" for this density and then  assert that a particular hierarchical clustering procedure returns finite  trees that converge to this cluster tree in some suitable sense. Recent work  by Eldridge et al has used a criterion called "merge distortion" to assess  the discrepancy between the target (infinite) tree and the tree estimated  from a finite sample. Specific algorithms have been found to be consistent in the sense of having the right limit, and their rates of convergence have been determined.  The present paper is interested in extending this methodology to case (2).  The starting point is, necessarily, to specify some kind of underlying  distribution over graphs. The authors make a very general choice: the  underlying distribution is a "graphon". Briefly, this is a generative process over (infinite) graphs specified by a function W: [0,1]^2 --> [0,1]. To sample a graph with n nodes: -- Pick n values x1, .., xn at random from [0,1]. Here, xi is a latent feature associated with node i. -- Put an edge between nodes i and j with probability W(xi, xj). There has recently been quite a lot of interest in this generative process, motivated by the kinds of large graphs that are central to computational  social science.  The authors define a natural hierarchical clustering associated with a  graphon: this is the obvious one, whose clusters are obtained by picking a threshold t, considering points x and x' "connected" if W(x,x') > t, and then identifying clusters with the resulting connected components. One  technical complication is to state this definition in a way that is not derailed by sets of measure zero, and the authors do so by setting up an equivalence relation in which measurable sets are considered equivalent if their set difference has measure zero.  With the limit object thus defined, the authors proceed to develop a notion of consistency. One complication is the latent variables: in any given graph, the latent variables associated with nodes are not specified in advance; thus they need to be marginalized out. This is the one key difficulty of case (2) (see above) that does not arise in case (1).  The authors then use a notion of merge distortion (as in earlier work) to define consistency.  Finally, they give an algorithm for estimating the graphon cluster tree that is provably consistent. It is based on first estimating the edge probabilities and then applying single linkage.
(See summary for main strengths of the paper)  There is some related work that the authors should reference carefully: the  paper of Steinwart on "Adaptive density level set clustering". Like the present  paper, Steinwart takes great pains to define connectedness in a way that is  resistant to sets of measure zero. Can the authors comment on how their approach  differs?  All in all, this is important, exciting, and novel work.


Dynamic Filter Networks ESAT-PSI, KU Leuven, iMinds Tinne Tuytelaars ESAT-PSI, KU Leuven, iMinds
In a traditional convolutional layer, the learned filters stay fixed after training. In contrast, we introduce a new framework, the Dynamic Filter Network, where filters are generated dynamically conditioned on an input. We show that this architecture is a powerful one, with increased flexibility thanks to its adaptive nature, yet without an excessive increase in the number of model parameters. A wide variety of filtering operations can be learned this way, including local spatial transformations, but also others like selective (de)blurring or adaptive feature extraction. Moreover, multiple such layers can be combined, e.g. in a recurrent architecture. We demonstrate the effectiveness of the dynamic filter network on the tasks of video and stereo prediction, and reach state-of-the-art performance on the moving MNIST dataset with a much smaller model. By visualizing the learned filters, we illustrate that the network has picked up flow information by only looking at unlabelled training data. This suggests that the network can be used to pretrain networks for various supervised tasks in an unsupervised way, like optical flow and depth estimation.

This paper introduces a new framework termed "Dynamic Filter Networks", where the filters of a convolutional layer are not fixed but are predicted from a small subnetwork. The experiments contain a 1) sanity check (or a unit test) of the proposed approach, and networks that 2) take 10 frames of moving MNIST digits from [19] and predict the next 10 frames, 3) that predict 2 future frames in a highway driving application, and 4) that predict stereo views, posed as generating right view given left. In particular the experiments on moving MNIST show that the model can perform this task better than previous work and with many fewer parameters than previous work [18,19].
First, I'd like to say that this paper is very nicely written and was easy to read and understand. I also appreciated the thoroughness of experiments showing results in 1) unit test, 2) MNIST toy data, and 3) a slightly more real application.  My major concern with this paper is with marketing the dynamic filter network as something entirely new. Certainly, the model predicts some parameters that happen to be used in a convolution, but that simply means that you have a slightly more complex mapping from inputs to outputs. The approach is still entirely parametric, you're merely tweaking the functional form of the mapping and allowing your network slightly more complex interactions. The authors devote a paragraph to this point on line 114, stating that:  ``` we make a distinction between *model parameters* and *dynamically generated parameters*. Model parameters denote the layer parameters that are initialized in advance and only updated during training. They are the same for all samples. Dynamically generated parameters are sample-specific, and are produced on-the-fly without a need for initialization. ```  this is incorrect. The dynamic filter network is exactly identical to every other normal network: it still has a fixed set of parameters that are initialized in advance and updated during training. You're merely designing a more complex mapping. The definition of a parameter is not "the `weight` field in a convolutional layer". It is merely a number that changes during backprop. The authors spend an entire section 3.3 on drawing analogies to STNs, ResNets, etc. and I find these very far fetched and abstract comparisons.  Finally, the experiments are nicely done and I appreciate Figure 7 that visualizes what the network learns.  In summary, I think the authors make a big deal out of a simple idea, market it in a strange way, but ultimately do a nice job in execution: writing, experiments, and presenting the results. I think the experiments in this paper nicely highlight the strength of exactly this kind of model, where predicting the filters of a conv layer dynamically makes a lot of sense architecturally. Overall I like the paper and I'd be happy to see it accepted.   Minor: - "Humans are good at predicting another view from related views". Are they? This is a poor way to start the intro of this paper. In general I'm not a fan of the first paragraph. - The authors have a strange interpretation of the Softmax function, claiming a few times that it helps "encourage elements to only have a few non-zero elements" (line 248). This is by no means the case. The network can choose to have high magnitude vectors where Softmax returns very peaky results, but it could also in principle return very diffuse vectors.


SoundNet: Learning Sound Representations from Unlabeled Video
We learn rich natural sound representations by capitalizing on large amounts of unlabeled sound data collected in the wild. We leverage the natural synchronization between vision and sound to learn an acoustic representation using two-million unlabeled videos. Unlabeled video has the advantage that it can be economically acquired at massive scales, yet contains useful signals about natural sound. We propose a student-teacher training procedure which transfers discriminative visual knowledge from well established visual recognition models into the sound modality using unlabeled video as a bridge. Our sound representation yields significant performance improvements over the state-of-the-art results on standard benchmarks for acoustic scene/object classification. Visualizations suggest some high-level semantics automatically emerge in the sound network, even though it is trained without ground truth labels.

This paper propose to use pre-trained CNNs for visual recognition to be used as a teacher to the learn a student sound network on the 2M acoustic unlabelled videos taken from fliker to learn natural sound representations.  The idea of transferring abstract knowledge learned from one modality to other in the student teacher network is interesting and looks appealing. Although the methodology used is not very novel it facilitate learning of much deeper sound-net. The convolution features learned are shown to give much better results then that of the hand crafted features thus the experimental results are encouraging.  The presentation of the sound net architecture is clear and most of the design choices are justified. Over all a well presented paper with incremental novelty but with good results.  The most crucial drawback of the paper is the absence of a reasonable baseline CNN which is trained on a large enough data-set without supervision. A simple soundnet based autoencoder as the unsupervised network should be trained on 2M training images as the baseline for acceptance of the paper.   My recommendation is a weak reject but I will be happy to change my rating if in the rebuttal authors can substantiate the utility of the teacher student setup of figure 1 over convolutional Auto encoder.
The most crucial drawback of the paper is the absence of a reasonable baseline CNN which is trained on a large enough data-set.  The largest labelled sound dataset used for evaluation is ESC-10/50 with 2000 sound recordings which is too small for training a CNN from scratch. In table 1a) all methods are said to be hand crafted features except that of [24] which is supervised and trained on 2000 sound recordings?   I believe that the source if improvement over [24] could be  (i) the difference in architecture: this can be simply ruled out by training sound-net on ESC as a replacement of [24] and show that the sound-net is too deep to be trained directly on such small datasets. (ii) The knowledge transferred from state of the art visual recognition networks (Alex-net/VGG) as claimed by the paper. (iii) Simply because of the fact that we have now a large unlabelled dataset.  Given over 2M soundbites at our disposal, a very simple baseline is a traditional convolution autoencoder with tied weights similar to sound-net encoder. This encoder can be fine tuned on ECS and iHear separately for comparison. (A simple classifier on the autoencoder codes can also be used). This experiment is mandatory to verify weather the vision to audio knowledge transfer is useful. i.e how much improvement you get from merely using huge unlabelled data-set and quantify the contribution of visual training.  Another very important question is that flicker is said to mined with keywords. Although, these keywords does not correspond to the classification labels in the test experiment why can't we learn a siamese network with these surrogate labels and then use finetuning?     Other minor comments: - a brief description of these baselines is needed as the reader is bound to refer to line 47-48 even to know that out of so many baselines only [24] is a conv-net for which the architecture is not outlined in the paper). - As iHear seems to be more diverse and biggest labelled data-set, all the baseline (especially [24]) should be tested also on iHear.  - Why other unsupervised and supervised deep learning methods apart from [24] is not compared against?


Crowdsourced Clustering: Querying Edges vs Triangles
We consider the task of clustering items using answers from non-expert crowd workers. In such cases, the workers are often not able to label the items directly, however, it is reasonable to assume that they can compare items and judge whether they are similar or not. An important question is what queries to make, and we compare two types: random edge queries, where a pair of items is revealed, and random triangles, where a triple is. Since it is far too expensive to query all possible edges and/or triangles, we need to work with partial observations subject to a fixed query budget constraint. When a generative model for the data is available (and we consider a few of these) we determine the cost of a query by its entropy; when such models do not exist we use the average response time per query of the workers as a surrogate for the cost. In addition to theoretical justification, through several simulations and experiments on two real data sets on Amazon Mechanical Turk, we empirically demonstrate that, for a fixed budget, triangle queries uniformly outperform edge queries. Even though, in contrast to edge queries, triangle queries reveal dependent edges, they provide more reliable edges and, for a fixed budget, many more of them. We also provide a sufficient condition on the number of observations, edge densities inside and outside the clusters and the minimum cluster size required for the exact recovery of the true adjacency matrix via triangle queries using a convex optimization-based clustering algorithm.

The paper considers the problem of clustering a partially observed graph and compares two types of queries: edge queries vs triangle queries. The problem is motivated by the task of clustering unlabeled items using crowdsourcing platforms. In this setting, an edge query reveals a random pair of items to a crowd worker for comparison whereas a triangle query reveals three random items and asks if any two of them, none of them, or all items are in the same cluster. The authors then try to answer the following question: for a fixed query budget, which query type can better retrieve the true underlying graph and therefore results in a more exact clustering?   They provide a few generative stochastic models for both query types. The models allow them to compute the probability that a random edge in the observed graph is in error. They show that under reasonable assumptions on the parameters of their models, triangle queries find more reliable edges. They provide empirical evidence by presenting results from several simulations and two real data sets. Their experiments show that clustering via triangle queries outperforms clustering via edge queries by finding more reliable edges and finding more edges for a fixed budget.
The authors propose a novel strategy for partially observing a graph that consists of disjoint cliques(clusters). As they mention in the paper, an important application of this problem is in crowdsourcing platforms where a fixed budget is given and we want to cluster a large set of items as precisely as possible. While I think the problem is interesting and potentially useful across a variety of applications, I have two concerns regarding the novelty and thoroughness experimental analysis.          Regarding novelty, although the authors provide some related work, I feel they could greatly improve on the comparison on two areas in particular: (1) the benefit of triadic comparisons in crowdsourcing and (2) the utility of sampling triangles for graph or adjacency matrix estimation. Regarding (1) the authors do mention related work but there is no experimental comparison with any of them.  Similarly for (2)"Adaptively Learning the Crowd Kernel" provides a method to estimate the adjacency matrix -- how does this compare to the current work?          Along this line the authors must either compare their approach to other approaches and give a clear explanation of why this is not possible. Further, the experiments do not leave me convinced that querying triangles is in general a better strategy. This is because of the way that authors compare costs of an edge query with a triangle query. In order to have a fair comparison between the number of edge queries and triangle queries that one can generate with a fixed budget, each query type must be associated with a cost. The authors suggest that the cost of a query can be measured by computing its entropy which is only available if the generative model of the graph is known. For real world problems, the authors use the average response time per query as the cost of a query. In section 3, they claim that empirical studies shows that the cost of a triangle query is approximately 1.5 times of an edge query, but there is no reference for this. Of course one can empirically estimate this ratio for a new data set by finding the average response time per query for each query type, but what if this ratio is very large and therefore only a few triangle queries are allowed with the given budget? It is not clear to me why querying triangles is a better choice in this scenario. On the other hand, the paper claims that the cost of a triangle query is never more than 3 times the cost of an edge query because each triangle query contains three edges. I am not convinced by this reasoning. It seems that they ignore the fact that identifying dependencies between edges makes the problem harder to solve but at the same time they are using the advantage of solving triangle queries which are more reliable than edge queries exactly because they consider edge dependencies.            I would suggest the authors compare the performance of edge queries vs triangle queries over a wider range of cost ratios to study if there is a turning point such that using edge queries reveals more information about the graph for a fixed budget(i.e., the cost of triangle queries are so high that we can only make a few of them). Followed by an empirical study that shows triangle queries are statistically unlikely to be so expensive.


Generating Long-term Trajectories Using Deep Hierarchical Networks
We study the problem of modeling spatiotemporal trajectories over long time horizons using expert demonstrations. For instance, in sports, agents often choose action sequences with long-term goals in mind, such as achieving a certain strategic position. Conventional policy learning approaches, such as those based on Markov decision processes, generally fail at learning cohesive long-term behavior in such high-dimensional state spaces, and are only effective when fairly myopic decisionmaking yields the desired behavior. The key difficulty is that conventional models are "single-scale" and only learn a single state-action policy. We instead propose a hierarchical policy class that automatically reasons about both long-term and shortterm goals, which we instantiate as a hierarchical neural network. We showcase our approach in a case study on learning to imitate demonstrated basketball trajectories, and show that it generates significantly more realistic trajectories compared to non-hierarchical baselines as judged by professional sports analysts.

The paper describes a specific instantiation of a deep memory network with an attention mechanism which was designed for hierarchical, long-term, action planing in complicated problems. The network is trained via behavioral cloning of expert trajectories and evaluated on a novel task: predicting basketball player behavior from captured real play-data. The different choices of the model architecture are studied in detail and thoroughly evaluated.
The paper is well written in general (aside from the minor stylistic issues mentioned below) and addresses an important problem: That of learning hierarchical behavioral policies from data. The chosen benchmark is interesting and is a welcome change from the standard benchmarks that are often used for control tasks/imitation learning.  The experiments presented in the paper are thorough and different model choices are studied in separation, allowing the authors to attribute the performance improvement of their model to concrete decisions made in the modeling. This is a big plus. The model formulation seems reasonable and the connection between hierarchical policies and draws from attention mechanisms for language modeling (a very reasonable idea in my mind). Aside from these clearly positive points the paper unfortunately also has a few issues. I will name them and then detail my criticism in turn. Overall, in my eyes, the paper could still well be accepted if the issues with the slightly confusing notation and the formulation of the loss function/training details are addressed (and the bad quality figures should obviously be replaced).   Related work details and citations: - You fail to cite related work from imitation learning/inverse reinforcement learning (IRL)/apprenticeship learning that could be used to solve the presented problem. In fact the presented algorithm can be interpreted as one instance of imitation learning to which the community refers to with the term "behavioral cloning". - In the related work you interchangeably use policy and planner, which at least to someone from the RL community seems somewhat odd. It also seems that calling DQN a "shallow planner" is a bit unfortunate terminology :).   Notation issues/Loss functions used: - In Equation (2) and (3) the auxiliary transfer function is is not conditioned on s_t but from Figure 3 and your network definition further below it seems that, obviously g is predicted from s_t and thus m_g should be conditional on it. Do you omit this simply because you assume it is an independent random event ? - Am I correct in assuming that the actual posterior derived in Equation (3) (and for the computation of which you discretize the state and action space) is never used during training ? It seems each part of the model is trained in a supervised manner and then fine tuning only occurs on the micro action level without computing the integral. Or does the probability density to which you refer in Equation (7) actually correspond to Equation (3) ? This is unclear from the context. - It also seems that suddenly between Equation (3) and Equation (7) the dependency on the history disappears. I presume this is just an error, but it is sloppy notation wise nonetheless. - My MAIN ISSUE with the loss function is captured by the footnote  you have on page 4. Although it is not completely obvious what the loss in Equation (7) is (see comment above) it indeed seems that it is symmetrical in the output of the micro and macro networks. Given that these operate on the same input, and there is no supervision on the macro-goals during fine-tuning, it seems that the objective function then has a trivial local minimum in which one of the networks just outputs a uniform distribution (unless you actually correctly integrate as in Equation (3) in which case I am not 100% sure about this). If this is the case then the only thing preventing the model from converging to this solution is the initialization and the fact that you choose small learning rates (as you write). I am not a big fan of the tactic to rely on the hope that the used, poor, optimizer will not find an obvious flaw in an objective function.  Experimental details: - I assume the correct thing to do during evaluation would be to sample from the posterior defined by Equation (3), but this is never mentioned.   If this is not what you do, what do you do ? Do you simply use the maximum action from the deterministic network output ? - While the task you evaluate on is interesting and fits the presented model well it would have been nice to see a comparison also on one more standard task in which one would expect hierarchical behavior to help, to facilitate comparison to existing work. This is less of a concern if the used data will be made available with the publication of the method.  Regarding the visualizations: - While it indeed seems that your model does not reproduce exact test trajectories it would have been interesting to also depict the closest trajectory in the training data (perhaps aligned with time-warping) to the one produced for the depicted test cases (hinting that the model does not merely learn to remember the training data). - You also could have included more visualizations for the baseline methods as a comparison (at least in the supplementary material).  Regarding the Human preference study: - You write that "During extrapolation, the other nine players do not animate.". This seems odd to me. First of all, I assume that you mean that the players are not animated in the visualization while they are still simulated for the network input (as the network should only be capable of dealing with such examples). Second, why is this a sensible idea ? Would this not heavily restrict the capabilities of the experts to tell apart a real, played out, strategy from a generated one ? I somehow doubt that a play can be extrapolated reliably by the experts just based on a few initial frames. I would rather have expected that you simply highlight the target player.  Regarding the evaluation of the macro-planner: - From the visualizations in Figure 6b it seems that the agent only rarely visits the actual macro goals. This is surprising to me. Could this be a result of the fine-tuning procedure (in which no macro labels are provided) and hint at the fact that the model indeed does not nicely separate macro and micro actions (as I would suspect it will not given the objective function used, see discussion above) ?  Regarding the Benchmark analysis: - The number of different model combinations studied here is nice and it allows for a concrete attribution of performance improvements to different modeling choices. - The evaluation presented in Table 2 has one main problem: You evaluate classification accuracy of the different studied models. However, given that you discretize the two-dimensional continuous action space into a large 17x17 categorical distribution I am not surprised by the generally low accuracies. While I understand that this accuracy is what you optimize during training it is definitely conceivable that a trained model might predict slightly different actions in each step which have almost the same effect as the ground truth actions but result in 0 % classification accuracy. As such, a different (perhaps distance based) measure would have been good to have in this table! - Tying into the question regarding deterministic/stochastic action choices above, it would have also been nice to have an evaluation of stochastic action selection here (which I assume is not what is presented in the table, although this is also not mentioned). - It also would have been better if you had splitted the test-data into several folds and reported mean and variances (or perhaps better medians with quantiles) in the table such that it would be possible to judge the statistical significance of the differences.  Minor issues: - The figures in the table are of terrible quality, making the model graphs hard to read. Please convert such graphics to vector graphics or at least include high resolution versions. I don't quite understand how you did not realize this before submission, did something go wrong in the submission process here ? - In-text citations are consistently in the wrong citation style. Please read the author guidelines and correctly use natbib \citet/\citep citations! - There are a few articles missing throughout the text and some other minor text mistakes. Here are the instances I found:   l.  22 "timescale low-level actions" -> "timescale of"   l.  33 "For instance, in Figure 1" here you are talking about the green "player: but that is not mentioned in the text   l.  68 "previous work which focus on" -> "which focuses on"   l.  96 "policy should simultaneously optimizes behavior" -> "optimize"   l. 176 "we used an simultaneously predicted the next 4" -> "and"   l. 184 "but instead learns the final output from a feature" -> "which instead"   l. 255 "may applicable" -> "may be applicable"


Safe Exploration in Finite Markov Decision Processes with Gaussian Processes
In classical reinforcement learning agents accept arbitrary short term loss for long term gain when exploring their environment. This is infeasible for safety critical applications such as robotics, where even a single unsafe action may cause system failure or harm the environment. In this paper, we address the problem of safely exploring finite Markov decision processes (MDP). We define safety in terms of an a priori unknown safety constraint that depends on states and actions and satisfies certain regularity conditions expressed via a Gaussian process prior. We develop a novel algorithm, SAFEMDP, for this task and prove that it completely explores the safely reachable part of the MDP without violating the safety constraint. To achieve this, it cautiously explores safe states and actions in order to gain statistical confidence about the safety of unvisited state-action pairs from noisy observations collected while navigating the environment. Moreover, the algorithm explicitly considers reachability when exploring the MDP, ensuring that it does not get stuck in any state with no safe way out. We demonstrate our method on digital terrain models for the task of exploring an unknown map with a rover.

The contribution of this paper lies in extending the work of safe exploration in Bayesian optimization in [22] to include reachability and returnability constraints within the state space.   The theoretical results appear to be sound. From the paper, it is not clear to me though what the non-trivial technical challenges are in extending the proofs and derivations in [22] to account for the additional reachability and returnability constraints.   It is also not clear to me how the trade-off of performance in exploring the maximum reachable safe set for computational efficiency for the proposed algorithm compares to that of [14] since there is no empirical comparison with [14] even though a similar setup to [14] is used. I’m also concerned about how sensitive the safe exploration performance of the proposed algorithm is to poorly estimated GP hyperparameter values which are not available a priori.   More detailed comments are available below.
NOVELTY  Lines 57-59: The authors say that “[22] considered a bandit setting, where at each iteration any arm can be played. In contrast, in this paper we consider an MDP, which introduces restrictions in terms of reachability that have not been considered in Bayesian optimization before” and “The main contribution consists of extending the work on safe Bayesian optimization in [22] from the bandit setting to MDPs.” What is the implication of such additional reachability and returnability constraints on the difference in the resulting technical proofs and derivations? Does it invalidate the proof technique in [22] or did the authors use a similar proof technique in [22] with additional non-trivial technical challenges to be tackled?    PRACTICAL SIGNIFICANCE  Lines 281-2: How do you obtain the values of the lengthscales and noise standard deviation hyperparameters, considering that the height distribution is unknown? If you set them manually, how sensitive are your results in terms of safety to different hyperparameter values? Can you justify why it is practical to do this for safe exploration?  The authors have used the same experimental setup as that in [14]. Though the authors have indicated that [14] is much more computational demanding than their proposed approach, it is not clear whether how the trade-off of performance in exploring the maximum reachable safe set for computational efficiency for the proposed algorithm compares to that of [14].  Line 78: Can the assumption of a deterministic transition model be relaxed in the proposed work? If so, what are the implications? Does [14] impose a similar assumption?  In MDP planning, besides the exploration aspect, there is also the exploitation aspect that is not accounted for nor discussed in this paper. Must the safe exploration in Algorithm 1 be performed separately first before exploiting the MDP for optimal planning? Or can the exploration be performed simultaneously with exploitation in planning like those in the following references (albeit without safety constraints/considerations):  C. K. Ling, K. H. Low, and P. Jaillet (2016). Gaussian Process Planning with Lipschitz Continuous Reward Functions: Towards Unifying Bayesian Optimization, Active Learning, and Beyond. In Proc. AAAI.  Marchant, R.; Ramos, F.; and Sanner, S. (2014). Sequential Bayesian optimisation for spatial-temporal monitoring. In Proc. UAI.   CLARITY OF PRESENTATION  Equation 1: The authors can give an intuitive explanation of how epsilon is related to noise omega_t and/or the GP predictive uncertainty of r due to limited measurements.  To avoid confusion, I recommend naming S_t in line 159 and equation 5 differently from that in line 152 because they are defined differently.  In Theorem 1, what value is delta set to?    Minor issues: Fig. 1: The notations appear to be inconsistent with that in equation 3. Missing bracket in first line of for loop in Algorithm 1. line 192: Should u_t(s) be u_t(s’) instead? line 194: has has?


Variational Information Maximization for Feature Selection
Feature selection is one of the most fundamental problems in machine learning. An extensive body of work on information-theoretic feature selection exists which is based on maximizing mutual information between subsets of features and class labels. Practical methods are forced to rely on approximations due to the difficulty of estimating mutual information. We demonstrate that approximations made by existing methods are based on unrealistic assumptions. We formulate a more flexible and general class of assumptions based on variational distributions and use them to tractably generate lower bounds for mutual information. These bounds define a novel information-theoretic framework for feature selection, which we prove to be optimal under tree graphical models with proper choice of variational distributions. Our experiments demonstrate that the proposed method strongly outperforms existing information-theoretic feature selection approaches.

The paper proposes an interesting algorithm for feature selection motivated by mutual information maximization. The authors attempt to show problems with current approaches that address the same problem (but I believe that their argument is flawed). The main idea behind their approach is to estimate a lower bound on the mutual information and use this estimate for a greedy feature selection. The paper describes a set of experiments showing clear advantage of the proposed method over other information maximization approaches to feature selection.
There are several important issues that I believe must be addressed  A. The authors make the following argument against current approaches.    Current approaches are optimal under a pair of assumptions.    These assumptions are hardly ever satisfied simultaneously.    Therefore current approaches are flawed.    There is a clear flawed logic here. The only thing shown is that current    approaches are not optimal. But this is clear even from the greedy    perspective. Their algorithms is also clearly non optimal in the general    case. I don't think this is such a big deal, except that the    authors spend a lot of text with figures, etc., to make this argument.  B. I believe the strength of the paper is the new algorithm, and the    experimental evaluation which shows its superiority.    Still, I would like to point out the following problems with the experiments.    B1. They compare the method only to other approaches to information        maximization, and not to other feature selection schemes.         In particular I was missing comparison to the FS, as described, for        example, in the work of Das and Kempe referenced in the paper.    B2. The methodology used by the authors (apparently following other work        in the field) is to performs the comparison with 3 nearest neighbors        and linear SVM. As explained below my concern is mostly with the linear        SVM. I was expecting the following two experiments:        1. An estimate of the maximized information showing (hopefully)           that the proposed method gives larger values.        2. Nonlinear SVM. Here is why.              There is an implied assumption that information maximization is better       than correlation based techniques (cf line 24).       So One would expect the features selected using information maximization       to outperform those selected by, eg, FS.       But this would clearly not be the case if the classifier is linear.       Example: if both x and x^2 are features, linear classifiers can use       both. But information maximization would most likely not select both.        This raises the possibility that an algorithm more similar to FS       would perform better with linear SVM even though it would produce       smaller mutual information.   C. I was missing references and discussion of work related to "Markov Blanket".  D. Typo in Table 1 caption. What is listed as d should be T.     E. The authors state (line 55) that optimal feture selection that maximizes    the mutual information is NP-hard. Could you please provide a reference?     (I believe the problem is much more difficult.)


Fast -free Inference of Simulation Models with Bayesian Conditional Density Estimation
Many statistical models can be simulated forwards but have intractable likelihoods. Approximate Bayesian Computation (ABC) methods are used to infer properties of these models from data. Traditionally these methods approximate the posterior over parameters by conditioning on data being inside an -ball around the observed data, which is only correct in the limit → 0. Monte Carlo methods can then draw samples from the approximate posterior to approximate predictions or error bars on parameters. These algorithms critically slow down as → 0, and in practice draw samples from a broader distribution than the posterior. We propose a new approach to likelihood-free inference based on Bayesian conditional density estimation. Preliminary inferences based on limited simulation data are used to guide later simulations. In some cases, learning an accurate parametric representation of the entire true posterior distribution requires fewer model simulations than Monte Carlo ABC methods need to produce a single sample from an approximate posterior.
Conventional ABC algorithms such as the above suffer from three drawbacks. First, they only represent the parameter posterior as a set of (possibly weighted or correlated) samples. A samplebased representation easily gives estimates and error bars of individual parameters, and model predictions. However these computations are noisy, and it is not obvious how to perform some other computations using samples, such as combining posteriors from two separate analyses. Second, the parameter samples do not come from the correct Bayesian posterior, but from an approximation based on assuming a pseudo-observation that the data is within an -ball centred on the data actually observed. Third, as the -tolerance is reduced, it can become impractical to simulate the model enough times to match the observed data even once. When simulations are expensive to perform, good quality inference becomes impractical.
We propose a parametric approach to likelihood-free inference, which unlike conventional ABC does not suffer from the above three issues. Instead of returning samples from an -approximation to the posterior, our approach learns a parametric approximation to the exact posterior, which can be made as accurate as required. Preliminary fits to the posterior are used to guide future simulations, which can reduce the number of simulations required to learn an accurate approximation by orders of magnitude. Our approach uses conditional density estimation with Bayesian neural networks, and draws upon advances in parametric density estimation, stochastic variational inference, and recognition networks, as discussed in the related work section.
2 Bayesian conditional density estimation for likelihood-free inference 2.1 Simulator-based models and ABC Let θ be a vector of parameters controlling a simulator-based model, and let x be a data vector generated by the model. The model may be provided as a probabilistic program that can be easily simulated, and implicitly defines a likelihood p(x | θ), which we assume we cannot evaluate. Let p(θ) encode our prior beliefs about the parameters. Given an observation x o , we are interested in the parameter posterior p(θ | x = x o ) ∝ p(x = x o | θ) p(θ).
As the likelihood p(x = x o | θ) is unavailable, conventional Bayesian inference cannot be carried out. The principle behind ABC is to approximate p(x = x o | θ) by p( x − x o < | θ) for a sufficiently small value of , and then estimate the latter-e.g. by Monte Carlo-using simulations from the model. Hence, ABC approximates the posterior by p(θ | x − x o < ), which is typically broader and more uncertain. ABC can trade off computation for accuracy by decreasing , which improves the approximation to the posterior but requires more simulations from the model. However, the approximation becomes exact only when → 0, in which case simulations never match the observations, p( x − x o < | θ) → 0, and existing methods break down. In this paper, we refer to p(θ | x = x o ) as the exact posterior, as it corresponds to setting = 0 in p(θ | x − x o < ).
In most practical applications of ABC, x is taken to be a fixed-length vector of summary statistics that is calculated from data generated by the simulator, rather than the raw data itself. Extracting statistics is often necessary in practice, to reduce the dimensionality of the data and maintain p( x − x o < | θ) to practically acceptable levels. For the purposes of this paper, we will make no distinction between raw data and summary statistics, and we will regard the calculation of summary statistics as part of the data generating process.

The paper is on likelihood-free inference, that is on parametric inference for models where the likelihood function is too expensive to evaluate. It is proposed to obtain an approximation of the posterior distribution of the parameters by approximating the conditional distribution of data given parameters with a Gaussian mixture model (a mixture density network).  The authors see the main advantages over standard approximate Bayesian computation (ABC) in that - their approach is returning a "parametric approximation to the exact   posterior" as opposed to returning samples from an approximate posterior (line 49), - their approach is computationally more efficient. (line 55)  The paper contains a short theoretical part where the approach is shown to yield the correct posterior in the limit of infinitely many simulations if the mixture model can represent any density. The approach is verified on two toy models where the true posterior is known and two more demanding models with intractable likelihoods.
Technical quality ---------------------  My ranking is due to the following:  1. The authors say that their approach approximates or targets the exact posterior (e.g. lines 50; 291).  I strongly disagree with the statement. The paper only concerns the approximation of the posterior after conversion of data to summary statistics. But the summary statistics are most often not sufficient so that important information is lost and targeting the "exact posterior" is not possible any more.  2. It is emphasized that the approximation can be made as accurate as required (line 51).  I suppose this becomes possible when the number of components and hidden layer in the network model is increased. Unfortunately, model choice does not seem to get discussed: How should the user choose the number of layers and components, taking in consideration that more data need to be simulated to fit more complex models?  3. While I appreciate theoretical justifications, Proposition 1 did not provide me with much information. Asymptotically, rejection ABC recovers the posterior too, under even weaker conditions, because the threshold (bandwidth) decreases to zero as the number of simulations increases (e.g. Section 3.1. in Blum2010). In that sense, also in rejection ABC, the approximation "can be made as accurate as required".  4. Regression adjustment is a classical method in ABC that deals with the increase of computational cost when the bandwidth epsilon is decreased (see e.g. Section 4.1 of the 2010 review paper by Beaumont). Some of the work is briefly mentioned toward the end of the paper, but no experimental comparison is performed. Such a comparison, however, would have been very important, at least because  - they are standard practice in ABC,  - they aim at producing (approximate) posteriors for epsilon \to zero, like the proposed method,  - there are straightforward theoretical connections to the proposed method (see below the comments on novelty).        Further comments:  1. In Fig1 (left), the more advanced algorithm 2 (MDN with proposal) seems to produce a posterior that is less accurate than the simpler algorithm 1 (MDN with prior). Would you have an explanation for that? Is it not worrisome that the more expensive algorithm 2, which builds on algorithm 1, is able to decrease the accuracy of the approximation?  2. In sections 3.3 and 3.4., the log probability (density?) that the learned posterior assigns to the true parameter value is used to assess the accuracy of the method. I think this measure could be possibly misleading because in many cases, in particular for weakly informative data, the true posterior is not centered at the data generating parameter (consider e.g. the posterior of the mean of a Gaussian). Furthermore, it does not measure the accuracy of the spread of the posterior. Getting the spread (e.g. posterior variance) right, however, is important, as pointed by the authors (line 22).  3. It is rather common to work with uniform priors on a bounded interval (e.g. [0,1]). Is it correct that the estimated posterior pHat in (3) would then be a truncated mixture of Gaussians? For truncated (mixture of) Gaussians    - exact normalization is not possible and numerically challenging in higher dimensions,    - the marginals are generally not truncated (mixtures of) Gaussians (see e.g. Horrace2005)  Can this be a problem for the proposed approach?  4. Eq (7) in the supplementary material shows that q_phi(theta|x_0)/pTilde(theta) equals the likelihood function (as the proportionality factor does not matter). Since pTilde is known and fixed in each iteration, does learning q_phi thus correspond to learning the likelihood function?   Novelty/originality ---------------------  I think the novelty claims of the paper are too strong.  Related work is mentioned in section 4 but the discussion stays on the surface and the proposed method is not placed well in the existing body of research. It is presented as a "new approach to likelihood-free inference based on Bayesian conditional density estimation". But it seems to conceptually belong to existing likelihood-free inference approaches.  1. ABC has been approached via conditional density estimation since more than 10 years (see e.g. section 4.1 of the review by Beaumont 2010). This connection resulted in the methods for regression adjustment. All these methods target the approximate posterior in the limit of epsilon \to zero.  2. When linear models are used, a pilot ABC run is needed to restrict the parameter space, which results in discarding simulations. For nonlinear models, however, all simulations can be used (see e.g. Fig 1 of Blum's 2010 paper in Statistics and Computing, reference 5 in your paper). This also corresponds to "\epsilon-free inference". Moreover, Blum used neural networks to model the relation between data (summary statistics) and parameters as well.  3. The 2012 paper by Fan, Nott, and Sisson that is cited as reference 8 is an epsilon-free inference approach based on conditional density estimation too, and it also makes use of mixture models. While Fan et al focus on approximating the likelihood, the work by Bonassi et al (2011), discussed by Fan et al, focuses on approximating the posterior.  4. Likelihood-free inference methods can be classified into parametric and nonparametric approximations (see e.g. section 3 of reference 11). Classical ABC algorithms correspond to nonparametric approximations where the "epsilon" plays the role of the bandwidth. Parametric approximations, by construction, do not require such a bandwidth (an example is reference 12). The proposed method is a parametric approach so that it is not surprising that it is "epsilon-free".  I agree that other researchers may not have used the particular neural network used in this paper. But it is unclear to me why the paper is presented as a "new approach to likelihood-free inference". It seems more like a technical difference to previous work, and its advantages relative to recent efficient likelihood-free inference methods are neither presented nor discussed.  References ------------  Blum2010: Approximate Bayesian Computation: A Nonparametric Perspective, Journal of the American Statistical Association, 2010, 105, 1178-1187   Beaumont2010: Approximate Bayesian Computation in Evolution and Ecology, Annual Review of Ecology, Evolution, and Systematics, 2010, 41, 379-406  Horrace2005: W. Some results on the multivariate truncated normal distribution, Journal of Multivariate Analysis, 2005, 94, 209-221  Bonassi2011: Bayesian Learning from Marginal Data in Bionetwork Models, Statistical Applications in Genetics and Molecular Biology, 2011, 10.   Update: ----------  Based on the explanation and promises in the author reply, I increased my scores from 2 to 3. I suggest to take the following points to heart when revising the paper.  - Please clarify that the "exact posterior" refers to the distribution of the parameters given the summary statistics and not the actually exact posterior. The expression "exact posterior" is misleading.  - Please clarify the difference to existing methods for likelihood-free inference that are based on (parametric) conditional density estimation, and update novelty claims accordingly.  - Please acknowledge that previous work on likelihood-free inference was able to greatly reduce the computational cost too (e.g. ref 11, 15).  - Please clarify why the approach is approximating the posterior rather than the likelihood function. In my opinion, Eq (3) shows that q_phi(theta|x_0)/ptilde(theta) must be proportional to the likelihood function. And thus by learning phi, you obtain an approximation of the likelihood function.


Learning to learn by gradient descent by gradient descent
The move from hand-designed features to learned features in machine learning has been wildly successful. In spite of this, optimization algorithms are still designed by hand. In this paper we show how the design of an optimization algorithm can be cast as a learning problem, allowing the algorithm to learn to exploit structure in the problems of interest in an automatic way. Our learned algorithms, implemented by LSTMs, outperform generic, hand-designed competitors on the tasks for which they are trained, and also generalize well to new tasks with similar structure. We demonstrate this on a number of tasks, including simple convex problems, training neural networks, and styling images with neural art.

Paper 1982: Learning to learn by gradient descent by gradient descent  An LSTM learns entire (gradient-based) learning algorithms for certain classes of functions, extending similar work of the 1990s and early 2000s.
Comments (mostly on related work):   Authors: "The idea of using learning to learn or meta-learning to acquire knowledge or inductive biases has a long history [Thrun and Pratt, 1998]."  But the intro to this reference is muddling the waters by confusing meta-learning (which is about learning the learning algorithm itself) and transfer learning, subsuming basically everything under "meta-learning," even standard back-propagation, because it can be applied to some data set, and then may learn new data points more quickly (so this is just standard transfer learning).    To my knowledge, the first work on learning general learning algorithms written in a  universal programming language was published in 1987:   J.  Schmidhuber. Evolutionary principles in self-referential learning, or on learning how to learn: The meta-meta-... hook. Diploma thesis, Institut für Informatik, Technische Universität München, 1987.  Authors: "This work was built on by [Younger et al., 2001, Hochreiter et al., 2001] wherein a higher-level network act as a gradient descent procedure, with both levels trained during learning. Earlier work of Runarsson and Jonsson [2000] trains similar feed-forward meta-learning rules using evolutionary strategies. Alternatively Schmidhuber [1992, 1993] considers networks that are able to modify their own behavior and act as an alternative to recurrent networks in meta-learning. Note, however that these earlier works do not directly address the transfer of a learned training procedure to novel problem instances and instead focus on adaptivity in the online setting."  This does not quite do justice to the previous work. Schmidhuber's 1993 meta-RNN was not an alternative to RNNs but an RNN able to run arbitrary computable weight change algorithms on the RNN itself. It could sequentially address any of its own weights and read or modify it (a bit like an NTM whose storage cells are actually fast weights). The whole system was differentiable end-to-end such that gradient descent could be used to search for a learning algorithm and also a meta-learning algorithm and a meta-meta-learning algorithm and so on - no restrictions. And Hochreiter's meta-LSTM really did "directly address the transfer of a learned training procedure to novel problem instances" (the words of the authors of the submission) in practice, because it was trained on certain quadratic functions to learn a learning algorithm for quadratic functions, and then a new quadratic function was presented in form of input/target pairs to the input units of the meta-LSTM, which generalized from the previous problem instances (which were functions) to the new ones, such that it could learn the new quadratic function 30 times faster than backpropagation or gradient descent, that is, it really learned by gradient descent to learn much faster than by gradient descent. Whether the resulting learning algorithm used something that's closely related to some fast version of gradient descent (or something more sophisticated) was not analyzed, to my knowledge. But the setup was general enough to permit all kinds of computable learning algorithms.  I think it is very important that the authors make very clear what's different to their own approach, provided there is a major difference at all.   I think the main difference is that the authors explicitly bias their meta-learning setup towards the learning of gradient-based learning algorithms (while the previous work did not have an explicit bias of this kind). This may be a good thing, at least in the context of the studied experimental tasks, and should be emphasized.  Authors: "Finally, Daniel et al. [2016] considers using reinforcement learning to train a controller for selecting step-sizes, however this work is much more constrained than ours and still requires hand-tuned features."   However, there was a much earlier much more general reinforcement learning system that learned to learn learning algorithms (written in a general programming language - no restrictions), namely the success-story algorithm SSA (called EIRA in an earlier  ICML publication):  M. Wiering and J. Schmidhuber. Solving POMDPs using Levin search and EIRA. In L. Saitta, ed., Machine Learning: Proceedings of the 13th International Conference (ICML 1996), pages 534-542, Morgan Kaufmann Publishers, San Francisco, CA, 1996. Based on:  J. Schmidhuber. On learning how to learn learning strategies. TR FKI-198-94, TUM, 1994.  J. Schmidhuber, J. Zhao, and M. Wiering. Shifting inductive bias with success-story algorithm, adaptive Levin search, and incremental self-improvement. Machine Learning 28:105-130, 1997.  This system could run arbitrary algorithms, including arbitrary learning algorithms and meta-learning algorithms affecting its own code, using SSA to favor "good" learning algorithms that optimize reward intake per time interval. It should be easy for the authors of the submission to identify the main differences to their own approach.   Authors: "We implement the update rule for each coordinate using a two-layer Long Short Term Memory (LSTM) network" ...   Did they use the original LSTM of 1997, or the LSTM with forget gates for the recurrent units which most people use?:  F. A. Gers and J. Schmidhuber and F. Cummins. Learning to Forget: Continual Prediction with LSTM. Neural Computation, 12(10):2451--2471, 2000.  "an NTM-BFGS optimizer, because its use of external memory is similar to the Neural Turing Machine [Graves et al., 2014]."  So it is like the fast weight system of 1992 which separates storage and control?  Authors: "Quadratic functions. In this experiment we consider training an optimizer on a simple class of synthetic 10-dimensional quadratic functions."  So this is like the experiments with quadratic functions of Hochreiter et al's 2001 paper. Did the authors perform a direct comparison to the 2001 system?   Conclusion:  Authors: "We have shown how to cast the design of optimization algorithms as a learning problem, which enables us to train optimizers that are specialized to particular classes of functions."   Well, this was already shown by the previous work up to 2001 mentioned above. I think the conclusion should focus on what's really new in this paper.   It seems to me that the novelty lies in the explicit bias towards gradient-based learning algorithms (while the previous work did not have an explicit bias of this kind). This is probably a good bias for the experiments presented in this paper, and should be emphasized.   And the additional experiments (besides those with quadratic functions) are cool, and the results are intriguing! I hope that this paper will help to rekindle interest in gradient-based learning to learn. It should be published, provided the comments above are addressed. I'd like to see the revised version again.


End-to-End Goal-Driven Web Navigation
We propose a goal-driven web navigation as a benchmark task for evaluating an agent with abilities to understand natural language and plan on partially observed environments. In this challenging task, an agent navigates through a website, which is represented as a graph consisting of web pages as nodes and hyperlinks as directed edges, to find a web page in which a query appears. The agent is required to have sophisticated high-level reasoning based on natural languages and efficient sequential decision-making capability to succeed. We release a software tool, called WebNav, that automatically transforms a website into this goal-driven web navigation task, and as an example, we make WikiNav, a dataset constructed from the English Wikipedia. We extensively evaluate different variants of neural net based artificial agents on WikiNav and observe that the proposed goal-driven web navigation well reflects the advances in models, making it a suitable benchmark for evaluating future progress. Furthermore, we extend the WikiNav with questionanswer pairs from Jeopardy! and test the proposed agent based on recurrent neural networks against strong inverted index based search engines. The artificial agents trained on WikiNav outperforms the engined based approaches, demonstrating the capability of the proposed goal-driven navigation as a good proxy for measuring the progress in real-world tasks such as focused crawling and question-answering.

This paper introduces a task used as a benchmark in evaluating language understanding agents. The task takes a website and converts it into a graph of individual pages as nodes with hyperlinks represented by edges. Final performance is measured by starting from a page node and reaching a desired target node as represented by a sentence contained within the page for that node. The authors use Wikipedia to instantiate a dataset for this task and evaluate several existing neural net approaches. they also extend this test to include an evaluation of the existing Jeopardy question answer benchmark.
Overall this is an interesting paper but its contribution is very broad rather than having a specific  deeper contribution. the primary contribution seems to be framing Benchmark tests in this web navigation framework however their needs to be more discussion of related work and related benchmark test for comparison when introducing a new benchmark. If this is intended to be the main contribution of the paper the authors could save space by moving discussion of the specific neural network approaches to an appendix. also when introducing this new benchmark it would help to have more in-depth analysis of the dataset and we're evaluating approaches succeed and fail by way of examples in the paper. it's nice that the tool to create this type of data set is being released but I'm skeptical that this framework will create meaningful data sets for anything outside of Wikipedia. again given Wikipedia is the focus more comparison on other evaluations using Wikipedia data are needed. In particular there is some recent work on question answering from Wikipedia tables that seems relevant from the deep learning community. the writing is fairly clear overall but some details are omitted due to packing in a wide variety of information into what could easily be a journal length paper.   The wording of constraints 3 and 4 is a bit confusing. It seems like 3 is imposing a maximum number of allowed hops but please clarify.   It seems a bit strange to use the number of sentences in a query as the way of defining its difficulty. Sentence lengths can vary widely especially when selected for highly representative sentences of pages. it would be nice to see metrics of query difficulty related to number of words.  While it isn't the main focus of this paper a feed forward network with single hidden layer seems obviously too simple for this task.  The task as framed isn't a very natural way to explore a resource like Wikipedia so it's not surprising that humans perform poorly. I'm not sure that comparing to human performance is even meaningful  other than to say humans aren't good at exploring Wikipedia in this way.  It would be nice to see more analysis on the Jeopardy test to understand what about the queries makes the proposed approach outperform standard search engines.  intuitively it seems that Jeopardy questions are intentionally difficult when  posed purely as search queries.