Learning Generative Models with Visual Attention
Attention has long been proposed by psychologists to be important for efficiently dealing with the massive amounts of sensory stimulus in the neocortex. Inspired by the attention models in visual neuroscience and the need for object-centered data for generative models, we propose a deep-learning based generative framework using attention. The attentional mechanism propagates signals from the region of interest in a scene to an aligned canonical representation for generative modeling. By ignoring scene background clutter, the generative model can concentrate its resources on the object of interest. A convolutional neural net is employed to provide good initializations during posterior inference which uses Hamiltonian Monte Carlo. Upon learning images of faces, our model can robustly attend to the face region of novel test subjects. More importantly, our model can learn generative models of new faces from a novel dataset of large images where the face locations are not known.

Summary:This paper proposes a deep learning based spatial attention mechanism based on a probabilistic generative model. The approach enables identifying novel objects in large images, and hence might allow for better exploitation of unlabeled, uncropped data.Main Comments:This paper takes a solid step towards being able to apply deep learning methods to large uncropped, unlabeled images. The motivation is compelling: previous DL approaches have required ‘curated’ training data that sticks the object essentially in the center with minimal clutter/occlusions—overcoming this limitation would truly enable learning from unlimited unlabeled data.The experimental section contains a number of interesting experiments verifying that the approximate inference method is working, and that HMC is still worthwhile beyond this. The network can perform some notable and rarely addressed tasks such as the ability to learn generative models from large images without labels, and the ability to shift attention by conditioning on various target stimuli.The paper claims that the algorithm runs in time O(1) given the size of the image. However initializations farther from the target image are likely to require more approximate inference steps, so there is an implicit dependence on image size. It could be interesting to plot the required number of steps as a function of image size to see this scaling. Also the window patch supplied to the conv net might also need to be increased for large images, so the scaling is not really O(1).The paper is clearly written and easy to follow.


Spectral Methods Meet EM: A Provably Optimal Algorithm for Crowdsourcing
The Dawid-Skene estimator has been widely used for inferring the true labels from the noisy labels provided by non-expert crowdsourcing workers. However, since the estimator maximizes a non-convex log-likelihood function, it is hard to theoretically justify its performance. In this paper, we propose a two-stage efficient algorithm for multi-class crowd labeling problems. The first stage uses the spectral method to obtain an initial estimate of parameters. Then the second stage refines the estimation by optimizing the objective function of the Dawid-Skene estimator via the EM algorithm. We show that our algorithm achieves the optimal convergence rate up to a logarithmic factor. We conduct extensive experiments on synthetic and real datasets. Experimental results demonstrate that the proposed algorithm is comparable to the most accurate empirical approach, while outperforming several other recently proposed methods.

Thank you to the authors for their thoughtful feedback.  I agree that there are pros and cons to both method-of-moments and optimization methods, and a detailed discussion of them is not needed in this paper (which, after all, aims to unite them!).  But since the goal is to analyze an algorithm for a specific, practical problem, I think it is important to at least make sure the assumptions -- the main one being that the Dawid-Skene model is the process actually generating the data -- are clear so that a reader can decide whether they are plausible for a given application.Finally, though the dependence on m may sometimes be more important than the dependence on n, I think it is important to explain this limitation when discussing optimality in the abstract and introduction, where the claims currently seem overstated.  Even weakened in this way, they will still be interesting claims backed up by strong results.--previous review below--In this work the authors propose a two-stage method for extractingconsensus labels from the noisy ones assigned by possibly unreliableannotators.  In the first stage, the method applies spectraltechniques under the Dawid-Skene model of label noise to obtainestimates of each annotator's label confusion matrix.  It then usesthese estimates to initialize an EM algorithm, which alternatelyupdates the label estimates and the confusion matrices to improvelikelihood.  The idea to initialize EM with spectral methods is notnew, but in this paper the authors claim to show that the result ofthe two-stage process is near-optimal in an asymptotic sense.Finally, experiments on several real-world datasets show that theproposed method performs comparably to the best existing method.As noted in the introduction, EM algorithms (like the one by Dawid andSkene) generally set out to optimize non-convex likelihood functions,which, being hard, makes it difficult to obtain any kind ofperformance guarantees.  The move to spectral methods allows forresults like Theorem 1, which shows that, given enough data, parameterestimates will converge to the true parameters.  However, there is atradeoff here, which is not mentioned: the guarantee only applies ifthe model is correct.  That is, if annotators have different confusiontendencies on different items, or if their performance drifts overtime, or if they in any other way deviate from the Dawid-Skene model,then Theorem 1 will likely not hold (nor will Theorem 2, in fact).This problem is much less severe for optimization algorithms, like EM,that have the ability to try and fit an imperfect model to real data;they may not have global optimality guarantees, but their objectivescontinue to guide them in settings where spectral theory simply fails.The absence of this consideration is at the core of my biggest concernabout the paper.  In attempting to prove the qualifications of amethod for handling the unreliability and unpredictability of thirdparty annotators, the paper assumes a high degree of regularity intheir behavior.  To paraphrase a saying, some models are useful, butall of them are wrong; I have mixed feelings about a theory thatassumes otherwise.  At the very least, a thoughtful discussion of thisissue would be a valuable addition to the paper.Having said all of that, I am excited about the direction the authorsof this paper have pursued.  In practice, the spectral+EM strategy hasbecome popular; presumably the idea is to get a "best of both worlds"kind of result, where the spectral method hopefully gives a goodinitialization for EM, saving time and improving the final result, butin any case is probably no worse than random initialization.  Becauseof this I find the idea of theory for the combined algorithm highlyinteresting.  I'm not sure, however, whether Theorem 2 really fulfillsthis promise.  In particular, the assumption that the input C matricesare within alpha of the truth (for alpha that is small compared to thetruth) seems to require that the first stage has done basically all ofthe work.  In fact, the E-step used to obtain result (a) of Theorem 2is just choosing the maximum-likelihood y using the given C.  ArguablyEM is not really needed at all for the theory.On the other hand, the experiments almost make the case for theopposite possibility: that EM does most of the work.  Whetherinitialized by the spectral method or simple majority voting, the twoEM methods perform about the same.  In some cases the spectralinitialization leads to immediate convergence, suggesting it has founda good initialization, but even when the initialization is bad, 50iterations of EM seem to bring it in line.  Perhaps, then, the mostpractical use of this method is as a time-saving technique; it wouldbe interesting to see the runtimes in that case.I found Theorem 3 a bit hard to parse; presumably the \hat y and \hatmu in the infima are supposed to be mappings from the observed data tothe quantity in question?  At first glance they appear to just beconcrete values, which makes the claims seem a little strange.  In anycase, I found the theorem itself interesting, but I'm not sure I agreethat it shows that the convergence rate is "near optimal" since itdoesn't address n, which is usually the default convergence clock andarguably has the more dubious scaling in Equation 9, with large powersof k and a hard-to-interpret singular value.  In my opinion this makesthe optimality claims in the abstract and intro somewhat overstated.


Low-dimensional models of neural population activity in sensory cortical circuits
Neural responses in visual cortex are influenced by visual stimuli and by ongoing spiking activity in local circuits. An important challenge in computational neuroscience is to develop models that can account for both of these features in large multi-neuron recordings and to reveal how stimulus representations interact with and depend on cortical dynamics. Here we introduce a statistical model of neural population activity that integrates a nonlinear receptive field model with a latent dynamical model of ongoing cortical activity. This model captures temporal dynamics and correlations due to shared stimulus drive as well as common noise. Moreover, because the nonlinear stimulus inputs are mixed by the ongoing dynamics, the model can account for a multiple idiosyncratic receptive field shapes with a small number of nonlinear inputs to a low-dimensional dynamical model. We introduce a fast estimation method using online expectation maximization with Laplace approximations, for which inference scales linearly in both population size and recording duration. We test this model to multi-channel recordings from primary visual cortex and show that it accounts for neural tuning properties as well as cross-neural correlations.

One of the chief concerns of systems neuroscientists is characterising how individual neurons respond to sensory stimuli. Since the stimulus space is often huge, data is always limited, and neurons are fundamentally noisy, the statistical challenges involved with this characterisation have spurned a vibrant field of computational neuroscience. This paper considers a particular form of this task, where recordings are made of local populations of neurons, at some intermediate point in the processing hierarchy in the brain (e.g. neurons within a cortical column, within a primary cortical sensory area). Such recordings are very common already, and also are growing in number and fidelity. The paper argues that under these conditions, an assumption can often be made that the neurons under study are receiving inputs from a smaller number of common channels, which are both stimulus-driven and subject to recurrent dynamics. The authors proceed to formalise this as a statistically-reasoned generative model for the observed neural responses (as spike counts), and present a Laplace/EM inference scheme to invert it given population data. The authors validate the inference on a simulated data set, and then fit it to V1 data.Overall, I think this is a well-motivated problem, and the solution the authors offer is both novel and interesting. The paper is clear and well-written, and makes a useful contribution to the field. There are a few issues here and there, which I list below.More importantly, I think there's some weakness in the scientific interpretations of these results that the authors could clean up. The construction the paper champions is essentially a beefed-up subunit model. Subunit models have a long and rich history in systems neuroscience, starting at least as early as Barlow & Levick (1964) or Hochstein & Shapley (1976) (perhaps we should include Hubel & Wiesel 1959). These are also having renewed interest as hierarchical models of feedforward processing, e.g. Mineault et al (2012). The first stage of the proposed model in Figure 1 is essentially a collection of nonlinear subunits. The authors do flag subunits in the second sentence of the introduction, but this link (and terminology) is dropped beyond this point. Now, of course, the beef this paper adds is important and useful, and I am a fan. However, I think that the systems neuroscience community would be more willing to engage with this work and build upon it if the subunit idea were carried through, especially because it captures an intuition about the feedforward processing circuits in the brain. And, in the absence of an argument to the contrary, I think it would be worthwhile to use the terminology of subunits to refer to this first stage within the model. Some of these subunits, of course, would not be carrying stimulus-driven information, only latent dynamical signals; this shouldn't be a big deal.Given that, I think there is some interpretation that needs to be made of the results of the simulation (Fig 2A-B especially), and the real data (Fig 4F especially). In the simulation, the ground truth (and recovered) subunits look like simple cells; in the real recordings, the recovered subunit space looks more like LGN inputs (albeit without a visible surround). As much as I rotate these latter recovered features in my head to get around the identifiability issue, I can't make the two examples match. By the time Section 4 has come around, the authors have set everything in place for a big reveal of simple-cell subunits, and it's rather disappointing that the results don't deliver. Moreover, it's not clear how the recovered features would generate the type and diversity of V1 receptive fields we're expecting. There are many potential reasons why this might be the case (e.g. sampling biases; all the units were simple), but the authors are in a better position than I to provide a reasonable explanation of this. To make things stranger, the first paragraph of the discussion seems to claim that the model can capture both simple cells and complex cells, and that it does. This was not the impression I came away with from Figure 4. Ultimately, perhaps the paper is principly about a model, but V1 is probably the most well-understood model testbed for neural systems identification, and there are expectations for the results here. If no explanation is given to intuitively validate the recovered features, readers will conclude that it doesn't recover what's expected here, so it won't work elsewhere.Quality: The paper is technically sound, with a couple of minor typographical errors listed below. Aside from the larger issue above, one lingering question I had was how to choose the number of dimensions in the first layer. The analysis in Fig. 3 (and surrounding text) seems like the right place to do this, but as far as I can tell, the number of stimulus-driven and stimulus-independent subunits were fixed at the ground truth. There should be some discussion of what happens when this is underestimated or overestimated. Figure 3 does not provide good enough evidence, as this assumes the ground truth (and the trends are hard to see in 3A).Minor errors:- eq. (2) and line 100-2: should $\vec{s}_t$ be the spike count vector $\vec{y}_t$, for consistency with eq. (3), and the typically implementation of the spike feedback kernel? Then this vector should also be integer-valued not real-valued. - line 183: C/D/d should be boldfaced (as matrix/matrix/vector), not scalar- eqn (7), LHS: the $\vec{w}_t^T$ should be a $\vec{h}_t^T$. Still, I think these equations could be omitted since the derivatives are pretty simple. Clarity: In general, the paper is written clearly and organised well. There were a few points throughout the text where I felt some extra clarification would make it read better:- eqn (2) and below: $\vec{D} \cdot \vec{s}_t$ -- I would suggest using a convolution symbol $\ast$ in place of $\cdot$ to convey its intuition. Also the constant $\vec{d}$ is not annotated.- line 115-117: There's some ambiguity about what refers to individual neurons, what refers to the local (recorded/columnar) subpopulation, and what refers to all of V1. There's at least one way of misinterpreting what is being said here. Individual neurons may care only about a stimulus subspace (hence STC). A local subpopulation will care about a larger stimulus subpsace than any one neuron (though not proportionally larger). The whole of V1 probably cares about a substantial portion of the whole stimulus space. The important aspect of *this* model is that it learns a basis for the local subpopulation's responses, which is assumed to be reasonably small. This is captured in the introduction to the paper, but is obscured here.- line 126/127, and subsequent: I found it confusing to talk about $\vec{B}$ as the concatenation of all /nonlinearity/ parameters, when $f$ could be a linear function. This is an instance where /subunit/ parameters would work better.- line 146: make this "where the parameters B = {...} includes"- line 166: "one infers the distribution of trajectories," -- insert $\vec{x}_t$ here. Likewise, start of line 168, "estimates" -- insert a $\vec{B}$ here. - line 172/173: the covariances $\vec{Q}_t$ of the latent states. Better to call this $\vec{\Lambda}_t$ or $\vec{\Sigma}_t$ or something like that, to avoid conflation with the covariance of epsilon_t. Line 181 is currently particularly confusing with both Q_t and Q.- further to that, I wasn't entirely clear what the posterior covariances $\vec{Q}_t$ should look like. This seems like it could be of some importance with the minibatch procedure. Also is it correct that the posterior covariance on the latent states only affects inference on the dynamical system parameters, and not on the subunit/nonlinearity parameters? Beyond this, I didn't feel like I got a good sense of how the information was combined across minibatches to update both sets of parameters in the M-step.- Fig 2C: wasn't clear exactly how the noise correlations were estimated without repeats (as for 4D).- Fig 2E: some info is necessary on how to interpret this plot, and what its relevance is. Same again for 4D and 5D. This appears to be playing the role of a validating result, but for Fig 4D and 5D I have no idea why this conveys as much.- line 272: I was amused by the precision in the computation time, but I wasn't sure how to extrapolate from this anecdote.Originality: tick, but some work needed to link to subunit literature (see above).Significance: tick, but some work needed to link to subunit literature (see above).


Sparse PCA via Covariance Thresholding
In sparse principal component analysis we are given noisy observations of a lowrank matrix of dimension n × p and seek to reconstruct it under additional sparsity assumptions. In particular, we assume here that the principal components v 1 , . . . , v r have at most k 1 , · · · , k q non-zero entries respectively, and study the high-dimensional regime in which p is of the same order as n.
In an influential paper, Johnstone and Lu [JL04] introduced a simple algorithm that estimates the support of the principal vectors v 1 , . . . , v r by the largest entries in the diagonal of the empirical covariance. This method can be shown to succeed with high probability if k q ≤ C 1 n/ log p, and to fail with high probability if k q ≥ C 2 n/ log p for two constants 0 < C 1 , C 2 < ∞. Despite a considerable amount of work over the last ten years, no practical algorithm exists with provably better support recovery guarantees. Here we analyze a covariance thresholding algorithm that was recently proposed by Krauthgamer, Nadler and Vilenchik [KNV13]. We confirm empirical evidence presented by these authors and rigorously prove that the algorithm succeeds with high probability for k of order √ n. Recent conditional lower bounds [BR13] suggest that it might be impossible to do significantly better. The key technical component of our analysis develops new bounds on the norm of kernel random matrices, in regimes that were not considered before.

The authors affirm a conjecture posed by Krauthgamer, Nadler and Vilenchik. regarding the performance of an algorithm for sparse PCA. Namely, they consider doing sparse PCA by (essentially) performing PCA on a thresholded sample covariance matrix. They show that the algorithm succeeds in recovering the joint support of the sparse components up to the threshold k = O(\sqrt(n)), where "k" is the sparsity index and "n" the sample size. This is believed to be the threshold achievable by polynomial-time algorithms. The result closes the gap k \in [\sqrt{n/log p}, \sqrt{n}] by showing an efficient algorithm that works in this regime.I find the results interesting and the paper well-written, modulo some typos and inconsistencies which I assume are introduced from translating a longer version to NIPS format.  I have a few concerns and questions:(1) The authors focus on recovering the union of the supports of the components in the multi-spiked case. First, this is not clear from the start until presented in the theorems and discussed in Remark 2.1. It is worth emphasizing this early on. I also don't quite agree with the content of Remark 2.1. The authors mention that given the joint support and "n" fresh samples, it is possible to consistently estimate the individual supports, because one can get consistent estimates of the individual components (v_q) once one restrict to the joint support. I agree that one can get consistency for v_q's by usual PCA restricted to joint support, but this is \ell_2 consistency. It is not clear whether thresholding these eigenvectors will give the correct supports. In particular, the problem seems to depend on the relative sizes of the supports and the number of components "r". (Consider many components having very small and very large supports).(2) The second half of assumption (A2) is a bit questionable as the authors point out later in Remark 2.3. It is better to move that remark closer to where the assumption is presented. I have no serious objection here, as the results are interesting even in the single component case. However, I want to point out that by symmetry (switching q and q'), the second half of assumption is effectively assuming |v_{q,i}/v_{q',i}| = \gamma. That is, the reverse inequality also holds by assumption.(3) Is the passage of the estimated eigenvector once more through the estimated covariance matrix really necessary? This is step (7) of Algorithm 1. What seems to be implied is that without it consistency is not achieved. In particular, merely thresholding the eigenvectors does not provide the correct supports. Is this true or is the analysis inconclusive in this regard? Any comments clarifying this issue is helpful. Also, you might point out that you are looking at a slightly modified version of what was proposed by Krauthgamer, Nadler and Vilenchik.Some minor issues:- Algorithm (1) appears before some of the quantities involved are introduced. It is better to move it further down.- p.3, in step 3 of rough algorithm, B should be defined in terms of \hat{v}_1 and not the population version v_1.- Algorithm 1, definition of G': the upper limit of the sum should be 2n.- p.4, Table 1 should be replaced with Algorithm 1.- p.4, ... provides bounds "on" the estimation error.- p.5, ... "a" kernel inner product random matrix.- p.5, in the first displayed equation \tilde{z}_i is not defined.


The limits of squared Euclidean distance regularization *
Some of the simplest loss functions considered in Machine Learning are the square loss, the logistic loss and the hinge loss. The most common family of algorithms, including Gradient Descent (GD) with and without Weight Decay, always predict with a linear combination of the past instances. We give a random construction for sets of examples where the target linear weight vector is trivial to learn but any algorithm from the above family is drastically sub-optimal. Our lower bound on the latter algorithms holds even if the algorithms are enhanced with an arbitrary kernel function. This type of result was known for the square loss. However, we develop new techniques that let us prove such hardness results for any loss function satisfying some minimal requirements on the loss function (including the three listed above). We also show that algorithms that regularize with the squared Euclidean distance are easily confused by random features. Finally, we conclude by discussing related open problems regarding feed forward neural networks. We conjecture that our hardness results hold for any training algorithm that is based on the squared Euclidean distance regularization (i.e. Back-propagation with the Weight Decay heuristic).

This paper extends work of [WV05] in an interesting and I think significant way.  The paper considers the following problem: suppose you have n examples and a class of n possible target functions, where we fill out the matrix M, M_ij = c_j(x_i), with random {-1,+1} values.  We now want to learn from a small sample.  A direct counting argument (as in [BEHW87]) says that O(log n) examples are enough to learn.  But suppose we require the algorithm to choose some embedding of data into R^n and then to use a linear predictor w where w is a linear combination of training examples (it must lie in the span). The work of [WV05] showed that Omega(n) training examples would be needed to get low error under square loss.  This work extends that to talk about any "C-regular" loss function, which just means that if the true label is 1 and you predict a negative quantity, your loss is at least C (and similarly if the true label is -1 and you predict positive).  Pretty much all reasonable loss functions are 1-regular. Interestingly, these lower bounds are stronger when M is random than if M is the Hadamard matrix.  Also, the statement of the main result (Theorem 3) ends up being very clean.Overall, the paper is in some ways a little "light" in that it essentially has one clean theorem and proof (as opposed to a collection of results and implications) but I like such papers myself.  And I think the one main result is a nice one.Question for the authors: is there any connection between your results and work showing that data of this type has no embedding where each target is a large margin separator?  E.g., to take a specific example I am most familiar with: the Balcan-Blum-Srebro paper "improved guarantees for learning via similarity functions" contains a result showing that for a Hadamard matrix, for any kernel there must exist some column/target j such that any LTF has high hinge loss at any margin significantly larger than 1/sqrt(n).  The analysis is based on (1) these targets are pairwise uncorrelated and therefore this class has high SQ-dimension, and (2) if there *was* a good kernel, then a *random* LTF would have a non-negligible chance of being a weak learner, violating the SQ lower bound.


The Noisy Power Method: A Meta Algorithm with Applications
We provide a new robust convergence analysis of the well-known power method for computing the dominant singular vectors of a matrix that we call the noisy power method. Our result characterizes the convergence behavior of the algorithm when a significant amount noise is introduced after each matrix-vector multiplication. The noisy power method can be seen as a meta-algorithm that has recently found a number of important applications in a broad range of machine learning problems including alternating minimization for matrix completion, streaming principal component analysis (PCA), and privacy-preserving spectral analysis. Our general analysis subsumes several existing ad-hoc convergence bounds and resolves a number of open problems in multiple applications: Streaming PCA. A recent work of Mitliagkas et al. (NIPS 2013)  gives a spaceefficient algorithm for PCA in a streaming model where samples are drawn from a gaussian spiked covariance model. We give a simpler and more general analysis that applies to arbitrary distributions confirming experimental evidence of Mitliagkas et al. Moreover, even in the spiked covariance model our result gives quantitative improvements in a natural parameter regime. It is also notably simpler and follows easily from our general convergence analysis of the noisy power method together with a matrix Chernoff bound. Private PCA. We provide the first nearly-linear time algorithm for the problem of differentially private principal component analysis that achieves nearly tight worst-case error bounds. Complementing our worst-case bounds, we show that the error dependence of our algorithm on the matrix dimension can be replaced by an essentially tight dependence on the coherence of the matrix. This result resolves the main problem left open by Hardt and Roth (STOC 2013). The coherence is always bounded by the matrix dimension but often substantially smaller thus leading to strong average-case improvements over the optimal worst-case bound.

The paper conveys a broad analysis of the convergence of the pertubed power method, called noisy power method (NPM), recently applied in a number of applications, which subsume computation of singular vectors of a matrix in a presence of noise. The authors infer and state an explicit general convergence rate of NPM subject to the noise of a bounded magnitude. Thereby, they ensure a global convergence of NPM, only local convergence of NPM being previously claimed for the matrix completion application. In the case of the streaming PCA, the generality of the estimates allows to extend the model to arbitrary distributions. Regarding the privacy-preserving spectral analysis, a fast NPM based algorithm is elaborated, which keeps a nearly optimal accuracy.Quality and Clarity: The paper is technically sound. The presentation of the developed theory is clear and cohesive.Originality: This work extends and extensively generalizes earlier applications of NPM imposing new theoretical findings regarding the convergence of the method.Significance: The work develops an important and valuable theory that has already shown to advance the applications of NPM.


Dependent nonparametric trees for dynamic hierarchical clustering
Hierarchical clustering methods offer an intuitive and powerful way to model a wide variety of data sets. However, the assumption of a fixed hierarchy is often overly restrictive when working with data generated over a period of time: We expect both the structure of our hierarchy, and the parameters of the clusters, to evolve with time. In this paper, we present a distribution over collections of time-dependent, infinite-dimensional trees that can be used to model evolving hierarchies, and present an efficient and scalable algorithm for performing approximate inference in such a model. We demonstrate the efficacy of our model and inference algorithm on both synthetic data and real-world document corpora.

The authors propose a dynamic hierarchical clustering model which allows hierarchies clusters (topics) and corresponding parameters (popularities, word frequencies) to vary in time. Indeed, it is a stochastic process with tree structured marginals so a different hierarchical clustering is specified per time index. Since in general it is a computationally expensive task to do full Bayesian inference they propose an approximate inference scheme where the parameters of each node are MAP estimates and the node re-assignment of observations is done by a Gibbs step. It is based on the tree structured stick breaking process so it allows the observations to be assigned to internal nodes and leaves of the tree rather than just leaves or complete paths. My main concern is that it is not fully exploiting the non-parametric nature of the model since the authors fixed the depth of the trees in the experiments. It would be nice that the depth varied over time or that a more detailed sensitivity analysis was presented.Some specific comments:Line 111: In this section I suggest to write the full generative model for each example (this could be added to the supplementary material if it is too long) just to be able to see how one could do full Bayesian inference in this model. Furthermore, maybe a MAP-Bayes like scheme (hard clustering assingment) could be obtained as a limit? I guess this would be a different approach so no need to include this.Line 244: Here it would be nice to have the corresponding conditional distribution for the parameters \theta for both examples in the previous section. It would be good to have this expression so one can see why you can't obtain MAP estimates directly and hence, the need for the coordinate ascent on this expression. Line 358: why is it sensitive to initialization? Or is this just done to ensure that the Gibbs sampling has converged?Line 371: while it is true that shallow hierarchies are easier to interpret it is still not quite satisfying to choose the depth of the tree to be 3 or 4 for every tree. One of the benefits of the model is to get different trees per time index so it would be nice that their corresponding depths varied as well. Furthermore, it would be nice to have a principled way of choosing the depth or at least a criterion which exploits the nonparametric nature of the model (in terms of tree distance, say). For example, as in Lakshminarayanan et al (2013), Mondrian Forests:Efficient Online Random Forests, ArXiv 1406.2673 where they suggest a budget parameter to handle this or in linkage algorithms where you choose your tree depth depending on certain criterion based on a fixed distance.Line 399: Is it possible to obtain a MAP tree? Since one can obtain a different tree per time index it would be interesting to know if there is a way to combine/summarize such trees just to see if there is a recurring pattern. This could justify or not the assumption that hierarchies vary over time.Supplementary material:Line 53: The proof of theorem 1 could be improved. It is not clear to me why is it the case that after deterministic deletion the resulting process is still TSSBP. The proof could be more mathematically precise.Quality: this is a really well written and structured paper.Clarity: the importance of the problem is clearly explained and the paper has nice figures to illustrate the concepts and results.Originality and Significance : the paper introduces novel ideas.


Algorithm selection by rational metareasoning as a model of human strategy selection
Selecting the right algorithm is an important problem in computer science, because the algorithm often has to exploit the structure of the input to be efficient. The human mind faces the same challenge. Therefore, solutions to the algorithm selection problem can inspire models of human strategy selection and vice versa. Here, we view the algorithm selection problem as a special case of metareasoning and derive a solution that outperforms existing methods in sorting algorithm selection. We apply our theory to model how people choose between cognitive strategies and test its prediction in a behavioral experiment. We find that people quickly learn to adaptively choose between cognitive strategies. People's choices in our experiment are consistent with our model but inconsistent with previous theories of human strategy selection. Rational metareasoning appears to be a promising framework for reverse-engineering how people choose among cognitive strategies and translating the results into better solutions to the algorithm selection problem.

This paper is refreshing in its scope and ambition; I've reviewed and read many NIPS papers that are just one tiny idea explored in excruciating detail. However, despite the paper's considerable breadth and the effort placed into evaluation, I feel this paper may walk the line too far to the side of broad and shallow, versus narrow and deep. I lay out specific concerns below that limit the impact of the results presented.The area of algorithm selection / metareasoning is key in both AI and cog sci. I am surprised that more work hasn't been done in this important area. (I don't know the literature, but I am trusting the authors' review which mostly consists of decade-old work. They do fail to cite work in cognitivearchitectures like SOAR and ACT-R which probably doesn't have a lot tooffer in terms of how people learn metastrategies.)I am confused by the authors' decision to focus on the domain of sortingalgorthms because this domain doesn't lend itself well to the rationalmetareasoning (RM) model. RM attempts to optimize a combined measure ofalgorithm accuracy and algorithm time/opportunity cost. However, for sortingalgorithms, don't all algorithms achieve perfect accuracy? And as a result,isn't the modeling of scoring (Equations 3, 8, 9, and 10) irrelevant?I also wonder whether there is much benefit of a Bayesian linear regressionapproach over simple ridge regression, given that only parameter means arebeing utilized (Equation 12)?  That said, the representation selected seemsvery sensible (using the two features, their logs, and 2nd order polynomialterms involving these features and their logs).The work claims to contribute both to state-of-the-art AI approaches toalgorithm selection and to the psychological theory of metareasoning. I havequestions about its contributions to each, which I'll address separately.With regard to AI, the authors compare their results to two existing models.The Lagoudakis method doesn't appear to utilize presortedness. As a result,it seems more like a straw man than a serious contender. With regard tothe comparison to Guo, I had some concern that (based on the text on line 160)a different measure of presortedness was being used by Guo, but the authorsassure me in their rebuttal that they use the same representation as Guo.With regard to the work's contribution to cognitive science:  In human strategyselection, the trade off between accuracy and opportunity cost is key, and theopportunity cost involved is not only the run time of an algorithm, but thecost of selecting an algorithm, as reflected in Simon's notion of satisficing.Thus, I question selection of sorting algorithms as the most suitable domainfor studying human strategy selection.  Although the experimental set upin Section 5 is elaborate and impressive, the coarse performance statistics(proportion of merge sort selection and overall quality of selection)hardly make a compelling argument that the RM model is correct.  All weknow from the experiment is that both the RM model and people are pretty good at selecting strategies, whereas the other models are not. This result gives us little insight as to whether the RM model is a good cognitive model.I couldn't find much detail about training the models used in Section 5, but ifthese are really meant to be cognitive models they should be trained on thesame data that people had available during practice, and the same total numberof trials.  The author rebuttal assures me that the training data is identical.MINOR COMMENTS:[093]: A Gaussian distribution may be acceptable for sorting algorithm runtimes, but it's probably not the best choice for modeling human responsetimes.  On a range of tasks from simple button presses to memory retrieval tocomplex problem solving, reaction times tend to have a long-tailed asymmetricdistribution.[091]: The standard deviation does not appear to be a polynomial in theextended features (Equation 7)[168]: The table caption should explain the performance measure. I believeit is the percentage of runs in which the optimal sorting algorithm wasselected.[295]: The text says that Equations 13 and 14 suggest the conditionsunder which merge sort will be chosen over cocktail sort. However, thecoefficients on n_comparisons and n_moves in Equation 13 are both smallerthan the corresponding coefficients in Equation 14, so it seems to methat cocktail sort should be chosen for all but the very shortest lists.


Expectation Backpropagation: Parameter-Free Training of Multilayer Neural Networks with Continuous or Discrete Weights
Multilayer Neural Networks (MNNs) are commonly trained using gradient descent-based methods, such as BackPropagation (BP). Inference in probabilistic graphical models is often done using variational Bayes methods, such as Expectation Propagation (EP). We show how an EP based approach can also be used to train deterministic MNNs. Specifically, we approximate the posterior of the weights given the data using a "mean-field" factorized distribution, in an online setting. Using online EP and the central limit theorem we find an analytical approximation to the Bayes update of this posterior, as well as the resulting Bayes estimates of the weights and outputs. Despite a different origin, the resulting algorithm, Expectation BackPropagation (EBP), is very similar to BP in form and efficiency. However, it has several additional advantages: (1) Training is parameter-free, given initial conditions (prior) and the MNN architecture. This is useful for large-scale problems, where parameter tuning is a major challenge. (2) The weights can be restricted to have discrete values. This is especially useful for implementing trained MNNs in precision limited hardware chips, thus improving their speed and energy efficiency by several orders of magnitude. We test the EBP algorithm numerically in eight binary text classification tasks. In all tasks, EBP outperforms: (1) standard BP with the optimal constant learning rate (2) previously reported state of the art. Interestingly, EBP-trained MNNs with binary weights usually perform better than MNNs with continuous (real) weights -if we average the MNN output using the inferred posterior.

The paper introduces a back propagation algorithm which instead of computing the gradient information, updates beliefs regarding posterior distributions of model parameters.  In doing so the learning algorithm relies on a mean-field approximation + the gaussian assumption (with the diagonal covariance matrix). They call this approach “expectation backdrop” (EBP).Overall,  I find some aspects of this work quite interesting. My questions / concerns / comments:1) The paper needs to provide a better discussion / comparison to the previous work on Bayesian learning of NNs ( see e.g., Radford Neal’s and David MacKay’s work from mid-90s + more recent work, e.g., on Langevin dynamics, ...). There have been a lot of work, and almost nothing is discussed here.2) The introduction of the EBP algorithm is motivated by arguing that the standard BP algorithm is not capable of learning NNs with binary weights which, according to the authors, much more efficient from the computational perspective at test time (however, one would need larger networks then?).   The EBP method is indeed capable of learning binary NNs in a fairly effective way. The catch is though that using binary models directly (i.e. the MAP solution) at test time directly does not seem to lead to competitive results (see column “Binary EBP-D” in Table 1: the error rate is  >= doubled for 4 datasets out of 7). What seems to work well is Bayesian model averaging with the binary model (“Binary EBP-P”) but this is again a ‘continuous’ computation which is at least as expensive as the computation with a normal NN.  At the very least the authors need to clarify this point, currently, as the binary version is the key motivation of this research (see the first paragraph of the paper).3) The notation in the model section needs to be cleaned up a bit. E.g., I was slightly confused with v vs. \nu (which is actually never formally introduced). The appendix was helpful but the paper needs to be understandable on its own.4) It would be interesting to see (empirically) how well this approach generalizes to deeper architecture (only models with a single hidden layers are considered in the paper).5) Since the argument is efficiently, I’d like to see a bit more discussion of this aspect in the paper (in the experimental section, e.g.). The approach is much more complex than standard BP (from the  implementation standpoint), and results even of the EBP-P version are not so obviously better than the ones for BP, so it would be nice to have more details to convince the readers. (However, the appendix does include some asymptotic analyses).


Probabilistic low-rank matrix completion on finite alphabets
The task of reconstructing a matrix given a sample of observed entries is known as the matrix completion problem. It arises in a wide range of problems, including recommender systems, collaborative filtering, dimensionality reduction, image processing, quantum physics or multi-class classification to name a few. Most works have focused on recovering an unknown real-valued low-rank matrix from randomly sub-sampling its entries. Here, we investigate the case where the observations take a finite number of values, corresponding for examples to ratings in recommender systems or labels in multi-class classification. We also consider a general sampling scheme (not necessarily uniform) over the matrix entries. The performance of a nuclear-norm penalized estimator is analyzed theoretically. More precisely, we derive bounds for the Kullback-Leibler divergence between the true and estimated distributions. In practice, we have also proposed an efficient algorithm based on lifted coordinate gradient descent in order to tackle potentially high dimensional settings.

** FROM AUTHOR FEEDBACK AND DISCUSSION WITH OTHER REVIEWERS **From discussion with other reviewers, the bounds in this paper are much better than previous results, but the difference and impact are poorly explained:BOUND AND IMPACT (from discussion)-- The squared Hellinger is dominated by Kullback, hence a bound with Kullback is stronger than Hellinger...-- ...the bounds proposed in this paper are really much better than previous results, but this is indeed badly explained in the paper. The bounds proposed in this paper are of order (assuming square d * d matrix to simplify) max( r d log d / n, r (log d / n) ^(1/2) ) where r = rank, d = number of rows = number of columns and n number of observed entries while previous bounds from [6] and [8] are of order r (d / n) ^ (1/2) Recalling that in matrix completion problem one typically has n >> d (otherwise no recovery is possible), this is a real improvement. Actually the result from this paper matches the results from Koltchinskii Tsybakov and Lounici (2011), that gives the best generalization errors for matrix completion to be found in literature. -- one doesn't need to know an upper bound for the rank, while [6] does.SUMMARY:Quality score updated to 6 due to novel theoretical contribution, after consultation with other reviewer (above).If the practical benchmark wasn't lacking (there are many standard baselines for MovieLens which are ignored), quality would be 7.If the paper discussed and illustrated the practical significance of T2, quality would be 8.------------------------------------------------MAIN IDEAS OF THE PAPERThe paper addresses the reconstruction of a matrix based on partial observations. The entries are discrete. It is assumed that the discrete entries are generated from a likelihood function f(X) that depends on matrix X (for the binary case) or p-1 matrices X_p for p categories. It is done by maximizing the likelihood over the observed entries (the same entry index can be observed more tha once, according to some \Pi distribution), but regularizing the model with a nuclear or trace norm constraint on X.As the true X is unknown, we would like to know how far off X^ is, in terms of the Kullback-Leibler (KL) divergence between f(X) and f(X^). For this two upper bounds are given. A version of the Rank 1 Descent (R1D) algorithm is given to solve for X^.The bounds are for 1-bit (p=2); experimental results are for the general case.The experiments are on a simulated data set, and MovieLens100K. In it, the paper shows that when the data is sampled from a low-rank model with logistic likelihood, the algorithm better recovers the true sampling probabilities than one that assumes a Gaussian likelihood instead of a logistic one. On MovieLens100K, which has ratings 1 to 5, the logistic model has a smaller predictive error than a Gaussian model.QUALITY / CLARITYThe paper seems to stand on two disconnected legs. An upper bound on the KL divergence between f(X) and f(X^) is provided in Theorem 2 for 1-bit matrix completion, but the paper does not explain or show its practical significance. It might be informative if the experiments have bearing on the developed theory, for instance, by including the KL bounds from T2 in Figure 1. Understandably, the bound relies on unknown constants, in which case a relevant interpretation of the bound is required.For these reasons I’m not sure the reader is adequately informed. After the main bound is stated, it is never stated, discussed, or evaluated.ORIGINALITYThe paper addresses a low-rank matrix completion for categorical random variables, which is a new contribution.There is a marginal extension of R1D (Algorithm 1).SIGNIFICANCEIt is not clear that the state of the art is advanced in a demonstrable way. There is a copious amount of literature evaluating 1 to 5 star rating prediction on the MovieLens and Netflix data sets, all of which are evaluated baselines that can be copied verbatim from known work.ADDITIONAL COMMENTS / SUGGESTIONSThe conclusion states that the rates of convergence in T2 are faster than that of Davenport and Cai's. I did not read their papers (Davenport's arxiv.org/abs/1209.3672) and Cai and Zhou (JMLR 2013) in extreme detail, but it seems that their bounds are in terms of Hellinger distance and Frobenius norm? It would be nice to know from the paper how the rates are faster. If there is a rebuttal period, will the authors please add this information? And clearly demonstrate how the bound advances the state of the art, or gives new insights?The binary case has been studied before outside the “1 bit” setting, for instance in [Matchbox: Large Scale Bayesian Recommendations; Stern, Herbrich, Graepel, in WWW 2009]. The likelihoods are similar to that employed in the paper (logit, probit), although the rank is explicitly constrained.Would the likelihood in the baseline evaluation (line 315) be the author's model (replacing p with X), except that a probit instead of logit link function is employed?For discrete ordinal ratings, a stochastic ordering is assumed in the input space, so that for four categories “awesome” > “good” > “mediocre” > “bad” are stochastically ordered [Regression models for ordinal data; McCullagh, in Journal of the Royal Statistical Society B, 1980]. It is a property that the model does not capture, but one that is very applicable to the MovieLens100K evaluation. See for instance [A Hierarchical Model for Ordinal Matrix Factorization; Paquet, Thomson, Winther, in Statistics and Computing, 2012] for ordered discrete alphabet and movie ratings.The paper could demarcate where / how R1D is extended.Lines 192 and 358: T2 suggests a setting for lambda.Line 410: Offsetting or shifting with a bias is common in the “Gaussian model” in literature: why not mean-centre the ratings? Analogously, it’s done for Principal Component Analysis.Typos, etc.Line 90: Shatten q-norm defined, never used (except q=1).Line 98: Hellinger distance defined, never used.Sections 3.1 and 3.2. Notation between e^j and h^j is switched around Line 220?Lines 219 and 250: Multiple definitionsLine 355: Figure 3.3Line 360: Table 3.3 and 3.3


Elementary Estimators for Graphical Models
We propose a class of closed-form estimators for sparsity-structured graphical models, expressed as exponential family distributions, under high-dimensional settings. Our approach builds on observing the precise manner in which the classical graphical model MLE "breaks down" under high-dimensional settings. Our estimator uses a carefully constructed, well-defined and closed-form backward map, and then performs thresholding operations to ensure the desired sparsity structure. We provide a rigorous statistical analysis that shows that surprisingly our simple class of estimators recovers the same asymptotic convergence rates as those of thè 1 -regularized MLEs that are much more difficult to compute. We corroborate this statistical performance, as well as significant computational advantages via simulations of both discrete and Gaussian graphical models.

I personally view this paper quite positively. I feel that the paper was well written, assumptions and theorems clearly stated, and (interestingly) the proofs quite simple and well written. On reading the reviewer discussion and author feedback, there seemed to be two main issues arising(1) Too similar to earlier papers? I think not - even if the high-level idea is similar, the devil is in the details, which need to be fleshed out to get the right rates of convergence, good performance in the experiments, and so on. If I gave someone capable the earlier paper and asked them to derive the kinds of results seen in this one, it would not be "straightforward" to extend those results - it really would need some hard work.(2) Too harsh assumptions? This might be true but does not warrant rejection - the idea of closed-formed estimators for these problems is *important* and the community deserves to be aware of these advances, especially because they have been shown to work very well practically in this paper - it really is a game-changer compared to graph lasso or even neighborhood estimation which needs lasso-solvers. I do think that further research will lighten the assumptions used (or get a better understanding of them).The experiments are very promising, this stuff actually seems to work fast and well, and so it is possibly a case of a gap between what works and what's provable. I suspect future theory developed by other people will weaken these assumptions - people will want to work on this or extend it etc because it seems to work well.In summary, I clearly stand on the side of acceptance.


Optimal rates for k-NN density and mode estimation
We present two related contributions of independent interest: (1) high-probability finite sample rates for k-NN density estimation, and (2) practical mode estimators -based on k-NN -which attain minimax-optimal rates under surprisingly general distributional conditions.

This paper presents a finite-sample analysis of k-NN density estimation. The paper begins by establishing uniform high-probability bounds for sup_{x \in R^d} | fk(x) - f(x) |, where fk(x) is the density estimate and f(x) the true density. The paper then uses this result to show that k-NN density estimation induces mode estimates that converge at an optimal rate.I enjoyed reading this paper, and found it to be clearly written. The technical development was well explained, making it possible to get a good feel for the main ideas used in the proof. The theoretical results also seem useful and relevant.Minor comments:- In Section 2.1, do any of the prior bounds hold simultaneously for all x in Rd? If not, it might be worth emphasizing this.- Lemma 2, I'm not sure \mathcal{F} has been defined.- Is there a reason lemmas 3 and 4 are not combined into a single result? To me, these two lemmas together form one of the most important results in the paper; I think if it were called a theorem it would be easier for the casual reader to take note.- On line 319, you say that "A_{x_0} is a level set." It is not clear to me that this is true given the assumptions made? Unless I'm missing something, you did not assume f(x) to be continuous outside of Ax0, so f(x') could get arbitrarily close to f(x0) for some x' that is far from x0.- The notation S + B(0, r) in Definition 6 should be defined.- Finally, the expressions "e.g." and "i.e." need to be surrounded by commas on both sides, e.g., like just there.


Altitude Training: Strong Bounds for Single-Layer Dropout
Dropout training, originally designed for deep neural networks, has been successful on high-dimensional single-layer natural language tasks. This paper proposes a theoretical explanation for this phenomenon: we show that, under a generative Poisson topic model with long documents, dropout training improves the exponent in the generalization bound for empirical risk minimization. Dropout achieves this gain much like a marathon runner who practices at altitude: once a classifier learns to perform reasonably well on training examples that have been artificially corrupted by dropout, it will do very well on the uncorrupted test set. We also show that, under similar conditions, dropout preserves the Bayes decision boundary and should therefore induce minimal bias in high dimensions.

-------------------------------------------review adjustment based on authors feedback-------------------------------------------In their response to the original review, the authors corrected my mistake and indeed the derivation is correct. Therefore, they have removed my main concern about this work. There are some issues that are left unclear and I hope the authors have a chance to clarify them. In (1) for example, the behavior of the RHS as a function of \delta is unclear. In their response, the authors said that this formula is correct only for \delta=1/2. Therefore, the discussion about the bias-variance tradeoff cannot be driven by this formula.------------------Original comments------------------Dropout seems to be an important component of the recent success of deep neural nets. This study attempt to shed some light on the reasons for this success. This line of research is of significance if it will allow us to come up with better regularization techniques. The authors study dropouts in the context of topic modeling where the data is assumed to be generated by a specific process. The main contribution of this work is a theoretical analysis that suggest that dropouts should increase accuracy. The paper is well written and contains novel contributions. The topic is relevant to the NIPS community. Unfortunately, there is a main component of the analysis that I fail to understand. In page 4, line 204 it is claimed that the variance after dropouts is (1-\delta) times the original variance (the same argument is repeated in page 11 line 593 of the appendix). However, IMHO, the variance is (1-\delta)^2 times the original variance. This is so since if X and Y are random variables such that Y= bX  where b is a scalar than the Var(Y)=b^2 Var(x). Am I wrong about it? Is it possible to fix the proof despite this issue?Another way to see this problem is looking at Figure 1b. It is implied from this figure that if X~N(\mu,\sigma^2) then the probability P[X < 0] is smaller than the probability P[Y < 0] where Y=(1-\delta)X. However, it is clear that these probabilities are identical as long as (1-\delta) > 0. I think that the presentation can be improved in (1) and (9). Following these formulas, in line 63 and in line 192 it says "the constants in the bound depend only on \delta". If a constant depend on \delta it is probably not a constant. This presentation requires the discussion in line 64 that explains why the bound does not behave as you may expect when \delta is close to one. Instead, making the dependency on \delta more explicit may be useful. One way to do it, that will keep the formula readable is to add a constant \Kappa to the R.H.S. in (1) and (9) and define \Kappa in a separate equation.In line 88 it is claimed that delta allows control over the bias-variance tradeoff. However, in line 81 it is claimed that the dropout does not introduce much of a bias. These claims seem to contradict each other. It would be helpful to clarify this issue.The theoretical analysis studies the zero-one loss but the experiments use the logistic loss (line 117). It would be nice to have some arguments to support these choices.In line 170 there seems to be a typo in the text: "the error on the original measure is actually much smaller than the error on the original measure"In (7) and (8) it is not clear if this is the training error or test error. In section 6 (Experiments and Discussion) there is no statistical significance analysis of the results. The plots don't contain error bars and therefore it is hard to tell if the results are artifact of randomness or do they represent a real phenomenon.


Beyond the Birkhoff Polytope: Convex Relaxations for Vector Permutation Problems
The Birkhoff polytope (the convex hull of the set of permutation matrices), which is represented using Θ(n 2 ) variables and constraints, is frequently invoked in formulating relaxations of optimization problems over permutations. Using a recent construction of Goemans [1], we show that when optimizing over the convex hull of the permutation vectors (the permutahedron), we can reduce the number of variables and constraints to Θ(n log n) in theory and Θ(n log 2 n) in practice. We modify the recent convex formulation of the 2-SUM problem introduced by Fogel et al. [2] to use this polytope, and demonstrate how we can attain results of similar quality in significantly less computational time for large n. To our knowledge, this is the first usage of Goemans' compact formulation of the permutahedron in a convex optimization problem. We also introduce a simpler regularization scheme for this convex formulation of the 2-SUM problem that yields good empirical results.

This paper studies the seriation problem where n objects with known characteristics must be ordered in a sequence such that a prescribed similarity measure of the characteristics decreases with the distance of the objects in the sequence. The noiseless seriation problem is formalized as the recovery of an R-matrix (whose entries are non-increasing as we move away from the diagonal horizontally or vertically) from a similarity matrix that has the structure of a pre-R-matrix (i.e., an R-matrix with permuted rows and columns). In the noisy seriation problem the similarity matrix is only approximately equal to a pre-R-matrix, and thus no exact R-matrix can be recovered.The noiseless seriation problem can be formulated as a 2-SUM optimization problem, which minimizes a convex quadratic form over all permutations of the integer vector (1,2,...,n). As the 2-SUM problem is NP-hard, this paper suggests to solve a convex relaxation of the 2-SUM problem over the convex hull of all permutations. It is argued that such a relaxation can also offer high-quality solutions for the noisy seriation problem. This approach requires a heuristic to recover a valid permutation from the solution of the relaxed problem.The convex hull of all permutations of (1,2,...,n) can be seen as a projection of the Birkhoff polytope of permutation matrices, which has a representation in terms of n^2 variables and constraints. This approach was adopted in reference [2]. The present paper suggests to use a more efficient sorting network representation of the convex hull of all permutations, which was discovered in [1] and only involves n*log(n) variables and constraints. While the noiseless seriation problem can be solved exactly and efficiently with a spectral method, the proposed relaxation method (as well as the relaxation method from [2]) provide good solutions for the noisy problem and when partial prior information about the sequence of the n elements is available (e.g., when some elements satisfy known precedence relations).The paper addresses an important topic with interesting applications in machine learning. The idea to use more efficient descriptions of the permutahedron is a good one, and the numerical results clearly demonstrate the merits of this approach. On the theoretical side, the contributions seem less convincing. It appears that this paper essentially combines an idea from [2] to solve the seriation problem by seeking convex relaxations of a related 2-SUM problem and from [3] to represent the permutahedron using a sorting network. Once these two ideas are combined, everything else is more or less straightforward. This is maybe the biggest weakness of the paper.I did not spot any mathematical errors or inaccuracies.The paper is generally well written and well structured. I just have some minor comments about the exposition:- l128: The sentence "Theorem 2.3 can be used to show that this construction can be used" sounds awkward.- l169: It would be useful to remind the reader that the Laplacian matrix is always positive semidefinite because it is symmetrical and diagonally dominant.- p4: It sounds as if the NP-hardness of the 2-SUM problem is the reason to study convex relaxations of (5). However, it is stated that (5) is efficiently and exactly solvable by the spectral method. This sounds like a contradiction. Of course, there is no contradiction, because the seriation problem is a special case of the 2-SUM problem where the input matrix is a pre-R-matrix, and this special case is tractable. It would be good to make this clearer and to motivate the formulations (7) and (9) by the desire to incorporate prior information and to account for noisy data (in which case the spectral method may no longer be applicable or display an inferior performance). It would be useful to work out these arguments more clearly. I had to read this several times until I (hopefully) understood the logic.- l199: I missed the definition of Pi_I. Is this the permutation matrix corresponding to the trivial permutation pi_I? So it is the identity matrix?- Figure 2(left): I am confused by the meaning of the outer circle and how it relates to the ellipsoidal level curves.- l291: "The asymptotically sorting network" sounds awkward.- l334: typo: "abd" -> "and".- Table 1: The caption states that solver times are averaged over 10 runs. However, the performance measures seem also to be averaged. Please clarify.


Constrained convex minimization via model-based excessive gap
We introduce a model-based excessive gap technique to analyze first-order primaldual methods for constrained convex minimization. As a result, we construct firstorder primal-dual methods with optimal convergence rates on the primal objective residual and the primal feasibility gap of their iterates separately. Through a dual smoothing and prox-center selection strategy, our framework subsumes the augmented Lagrangian, alternating direction, and dual fast-gradient methods as special cases, where our rates apply.

The paper proposes a primal dual optimization scheme for decomposable problems with linear coupling constraints, which can recover augmented Lagrangian, ADMM and fast dual descent methods.The paper is novel and highly non-trivial, but incremental in the sense that it provides convergence rates for the iterates themselves, avoiding the need to average as in existing methods. Unfortunately, the paper is hard to read due to the many technical details in the main part. Meaningful experiments are provided in the supplementary material only. The presented theory only applies with known accuracy for the subproblem solutions in each iteration, however the experiments (e.g. 1P2D) only use a few inexact steps.* The supplementary material contains a large collection of examples of applications, and these experiments seem encouraging for the presented method. Still, for all applications I do miss a comparison to the naive approach of just penalizing ||Ax-b||^2 in the objective and running standard accelerated gradient methods. Also, the experiments do not compare to other amenable methods, only standard ADMM in some cases.Instead of including just one experiment of the 1P2D method in the main paper, compared to the theoretical bound, it would be much more valuable to have some practical experiments comparing to competing methods, such as e.g. the SVM or Robust PCA case as presented in the appendix currently.* Another critique point I have is that the iteration complexity of the proposed methods are not discussed in the paper. While the theory only holds for known accuracy for the sub-problems / prox-operators, the experiments are performed with arbitrary inexact subproblems (12) or (15) or prox operators, e.g. a few FISTA iterations, which is not covered by the presented theory.(Update after author feedback: Unfortunately the feedback has not precisely addressed the computational complexity or the accuracy level of the subproblems. Hope this will be made more precise in the paper in case of acceptance.)* Structural assumption of decomposability:It is important to motivate the considered class of optimization problems. The assumption (3) alone is strong as it requires both sum structure as well as dependence of each summands on only one corresponding coordinate block. Using auxiliary variables, many ML problems can be transformed to form (1), which is however only explained in the lengthy supplementary material. Please add in the text that this is indeed possible for applications of interest. Similarly: When mentioning "we employ (3) to do away with the X -feasibility of the algorithmic iterates and focus on linear constraints": add a comment which important applications then still fall into the setting. The relation to Nesterov [1] could be made more clear also. Furthermore, it should be stated early that standard gradient methods do not apply to problems of the form (1) (even assuming prox tractability) due to the linear coupling constraints. However, they do apply for the penalized versions when just adding ||Ax-b|| to the objective.The abstract is not written precisely as to reveal the problem setting. You should clarify that your method can deal with linear coupling constraints, but assumes separable objective.* Introduction paragraph on comparison to [3]: This seems the wrong reference, I see no connection of the concepts used there to optimization setting here.* Adaptive proximal-centers are not explained, and it's not clear if the presented theory applies to this case as well.* Add references when mentioning "immediately supports parallel and distributed implementations in synchronous hardware architectures", as this is non-trivial.### Minor details:-l43: you might want to add that x_i \in X_i, or that the parts of coordinates x_i of x form a partition. And maybe use cartesian product notation instead of pi product for X.-l106: slightly tune down "radically different"-l118: +more structured-l201: phi is not defined here when introducing Nesterov's setting-ref [17]: venue missing


Poisson Process Jumping between an Unknown Number of Rates: Application to Neural Spike Data
We introduce a model where the rate of an inhomogeneous Poisson process is modified by a Chinese restaurant process. Applying a MCMC sampler to this model allows us to do posterior Bayesian inference about the number of states in Poisson-like data. Our sampler is shown to get accurate results for synthetic data and we apply it to V1 neuron spike data to find discrete firing rate states depending on the orientation of a stimulus.

In reply to the author's feedback "Our description of uniformization might be misleading, because the problems we describe do not occur in all of its applications. For the SGCP model discussed in our paper, however, the uniformization really is over the rate, which is lambda in our model. There is no MJP in the SGCP model, because the rate is continuous." After rereading the relevant sections of the paper, I am sure that this is incorrect. From page 2 line 070 "in our model lambda(t) is a latent piecewise constant process. ... Let us assume we already have i segments [from the piecewise constant functino lambda(t)] and draw the next jump time from an exponential distribution with rate f". Therefore, the authors' process is an MJP with constant rate f, and state space given by the positive reals. The emission model of the MJP is a Poisson process with rate lambda(t). To use uniformization in this case, we must upper bound the rate of the transitions of the MJP --- that is done by chosing f. Surely, lambda(t) might be large during bursts, but f is still constant and so the number of auxiliary uniformization events does not depend on lambda(t). "For our neural data the maximum rate would have to be the spiking rate during the strongestbursts, but this would lead to a very large number of (later thinned out) event times to be sampled in the long periods between bursts, which slows down the algorithm severely." That is incorrect. The maximum rate used by uniformization is f (that's the transition rate of the latent piecewise linear process) which is constant and doesn't depend on the spike rate.---In the manuscript "Poisson Process Jumping between an Unknown Number of Rates: Application to Neural Spike Data" the authors provide a model for spike train data using an Markov jump process (MJP) modulating the rate of spikes. The state space of the MJP is given by a Dirichlet process (DP) supported on the rate of spikes. The DP is then marginalized, and Metropolis-Hastings updates are derived for the resulting CRP. The authors apply this model to simulated data and also bursting in V1 neurons of cats (data from NSF-funded CRCNS Data Sharing repository). Overall, I thought that the manuscript was good. I did however think that it could have made more contact with computational neuroscience literature. Also, the manuscript incorrectly describes uniformization and the implications for efficiency.The authors' approach is good: it applies modern machine learning to a very concrete problem. The manuscript is therefore topical for the Neural Information Processing Systems conference. However, I felt like this paper could have been more in touch with classical work on HMMs in computational neuroscience. For example, I felt that the authors should have extended their model to ensemble cases. HMM methods have long been used in computational neuroscience to model spike rates (see Abeles, M., & Gerstein, G. L. 1988. Detecting spatiotemporal firing patterns among simultaneously recorded single neurons. Journal of Neurophysiology, 60(909), 909–24. for example.) Rather than focusing on single neurons, each HMM state in Abeles & Gerstein 1988 is parameterized by a separate rate for each neuron in the ensemble. Almost every other work on HMMs in computational neuroscience also looks at ensemble cases.The same model presented . was used in Vinayak Rao's PhD thesis for modelling neural spikes (chapter 5.4.4 of MCMC for continuous-time discrete-state systems, University College London, 2012). In that work, a Dirichlet process prior was put on the spike rates of neurons in a grasshopper auditory cortex, and then the rates and transitions were learned using uniformization. Note that this work was not published outside of his thesis and also his thesis does not go into much detail about the paradigms used, so I would definitely not be opposed to anyone publishing a detailed account of it.The authors do mention uniformization as another approach to this sort of data (page 7 lines 351-365), but they state that it would be inefficient because uniformization requires an upper bound on the rate, and this upper bound would be large due to bursting. Unfortunately, this is a misunderstanding of the way uniformization works and it's not the case that uniformization would be inefficient, at least not for this reason. The upper bound required in uniformization is on the transition rate of the underlying MJP, not on the likelihood parameter (the spike rate). The author's model, viewed as an MJP, has transition rate f, which is a parameter they mention on page 2 line 78. Because f is constant over the process, an upper bound on the transition rate of the MJP is given by f, and this (not the spike rate) would be used as an upper bound in uniformization. I do agree that it is confusing because there are two rates here: (1) the transition rate of the latent piecewise linear function (which is f) and (2) the spike rate (which is lambda(t)), but the upper bound in uniformization cares only about (1) and does not care about (2).I found that the comment by the authors that the shifting, adding, removing and switching updates alone were "very unlikely to reduce the number of states" to be sort of strange. If those actions are balanced, then around a mode shouldn't they be just as likely to reduce the number of states as to increase the number of states? Is the issue that these updates alone won't mix and therefore won't get near a mode? Would be nice to make that more explicit.Another comment that I found sort of strange was "The amounts of both jumps and states seem to be captured well, but for a large number of distinct states the mean seems to underestimate the true value. This is not surprising, because the lambda parameters are drawn from the same base distribution. For a large number of states the probability that two states are very similar becomes high, which makes them indistinguishable without observing more data" (page 5 lines 256 to 260) which the authors wrote to explain why the emperical statistic of the posterior number of states in Figure 3 (left) falls below the y=x line in the qq-plot. While it is true that the model will be unable to distinguish between lambda values that are close together with a small amount of data, it is also true that due to sampling noise (MCMC variance) a sampler will also occasionally put data that is close together and well explained by one cluster into two (or more) separate cluster, and the rate at which this happens should sort of cancel out the rate of `incorrectly' clustering data from more than one (very similar) clusters into one cluster. Due to the main theorem behind MCMC, under the procedure: 1. sample parameters y from the prior, 2. sample data x|y, 3. sample Y|x through your MCMC scheme, the variable Y should be an unbiased estimator of y. So, unless I am missing something, it seems the mismatch in the qq-plot in Figure 3 (left) is still unexplained and could indicate failure to mix, or a bug.Minor notes:The citations within the text of this manuscript are not conforming to the NIPS standard. NIPS should cite references using [1], [2], [3] etc, and then have [1], [2], [3] preceeding the citations in the reference section. While this might seem pedantic, it should be changed for the camera ready copy should the paper be accepted, in order to allow the entire NIPS proceedings to maintain it's well deserved familiar and professional feeling.


Bounded Regret for Finite-Armed Structured Bandits
We study a new type of K-armed bandit problem where the expected return of one arm may depend on the returns of other arms. We present a new algorithm for this general class of problems and show that under certain circumstances it is possible to achieve finite expected cumulative regret. We also give problemdependent lower bounds on the cumulative regret showing that at least in special cases the new algorithm is nearly optimal.

Paper Summary:This paper treats a general multi-armed bandit problem in which the mean reward of each arm depends on a common unknown parameter. The authors consider a simple modification of the UCB1 algorithm. They show, unsurprisingly, that the algorithm satisfies a regret bound like that of UCB1. The main improvement of this paper is to show when the optimal arm can be identified perfectly by samples of the optimal arm, algorithm’s regret is bounded by a constant independent of the time horizon.Overall: The bounds provided are pretty strong, and the proofs in the body of the paper are clear and well organized. The explicit finite time analysis here is novel. My main issues with the paper are (1) it's already known in the literature exactly when finite regret is possible and (2) I worry that the algorithm that is considered is not appropriate for more practical structured bandit problems. If the paper is accepted, I think that these two points need to be acknowledged clearly. Quality and Clarity: This paper is well executed. The bounds provided are pretty strong, and the proofs in the body of the paper are clear and well organized. A few minor typos are listed at the bottom of this review. Originality[1] showed that finite regret is possible under the same condition as in this work, but otherwise regret must grow logarithmically with the time horizon. That paper is focused on providing an asymptotically exact characterization of regret and makes the technical assumption that the parameter space is finite. It’s mostly dismissed by the authors for this reason. But Graves and Lai later removed the condition that the parameter space is finite. There are also four existing papers on permutation bandits, a special case under which finite regret is possible.  In addition, the algorithm considered seems to have also been studied before, for example in [Mailard and Mannor; Latent Bandits; 2014]. The main original contributions are (1) to show that a simple modification of UCB is enough to attain finite regret, and (2) to provide an explicit finite time analysis of how large this finite regret may be. -Todd L Graves and Tze Leung Lai. Asymptotically eﬃcient adaptive choice of control laws in controlled markov chains. SIAM journal on control and optimization, 35(3):715–743, 1997Significance:I worry that the improvements are mostly relevant for some very simple structured bandit problems, and the algorithm considered is not appropriate for more practical structured bandit problems.The proposed algorithm is a very simple modification of UCB1. It maintains independent confidence intervals for each arm, and constructs a confidence set containing the parameters consistent with these intervals. This is nice, because it’s interesting that a simple modification of UCB is enough to attain finite regret. On the other hand, the proposed algorithm is in some ways a major step backward relative other UCB-style algorithms for problems with dependent arms. Consider, for example, a linear bandit problem. Even if no single arm were sampled many times, the confidence sets of (e.g. [Dani et. al; Stochastic linear optimization under bandit feedback;  2008]) could shrink around the true parameter because there is a known relationship among the arms. This is also true of algorithms in [10,18]. The confidence sets in this paper don’t fully exploit the known structure of the problem and therefore don’t share this property. Minor comments:-Lines 35-38: “This model has already been considered when the dependencies are linear by [17], but not as far as we are aware for the general non-linear case.” You actually cite two papers [1, 18] that consider the general non-linear case, so this is a strange comment. There is also a paper by Graves and Lai, and a Thompson sampling paper by Gopalan, Mannor and Mansour in ICML this year. - The motivating example in lines 45-49 is a strange choice given the preceding pitch, since arms are modeled independently in this example. - You sometimes slip up and write the true parameter as \theta instead of \theta^*. See for example Theorem 2 and its proof, or the proof of Lemma 5. - Similarly, you sometimes slip up in your notation and use \mu _i to denote the true mean reward of arm i when really this should be denoted by \mu_i(\theta^*). See e.g. lines 92-93.-In remark 4 you refer to a regime where n is large, but the preceding bound has no dependence on n. - This is a picky comment… in lines 240-241, you don’t assume concentration inequalities. You assume rewards are sub-Gaussian, which implies concentration inequalities that you can then reference in this step. - When referencing Lemma 5, you often use the expression given after equality ( e), so this should be in the statement of the result and not just the proof. - Line 272, equality (b) is an inequality. - In the proof of Lemma 7, given the current problem formulation it seems like theta-tilde and \theta are completely arbitrary. I believe you mean to specify that theta-tilde is in the confidence set, and to write theta^* instead of theta. -In line 324, the “if” after (b) is a typo.****************************************Post-rebuttal comments:****************************************This is a well executed paper that makes several technical contributions. I think it should be accepted.Let me apologize. The Lai + Graves paper doesn't seem to explicitly state that regret is finite, only that its growth rate is sub-logarithmic. So regret could grow as log(log(t)) for example. Still, the authors should clarify that the condition for finite regret -- "choosing the optimal arm is sufficient to prove that it is indeed optimal" -- plays a very important role in past work. Were this a fundamentally new insight, I would have given the paper a very enthusiastic review.


On the Convergence Rate of Decomposable Submodular Function Minimization
Submodular functions describe a variety of discrete problems in machine learning, signal processing, and computer vision. However, minimizing submodular functions poses a number of algorithmic challenges. Recent work introduced an easy-to-use, parallelizable algorithm for minimizing submodular functions that decompose as the sum of "simple" submodular functions. Empirically, this algorithm performs extremely well, but no theoretical analysis was given. In this paper, we show that the algorithm converges linearly, and we provide upper and lower bounds on the rate of convergence. Our proof relies on the geometry of submodular polyhedra and draws on results from spectral graph theory.

Eq. (3): what if the maximization problem is unbounded?Line 216: the phrase "intimately related" suggests that $\kappa^*$ completely determines the convergence rate. But what Theorem 2 shows is that $\kappa^*$ only upper bounds the convergence rate, which is a weaker property.Line 324: "the analysis in this case subsumes the analysis in the general case". Can you elaborate this argument?I am curious, would you get better bounds if you assumed that functions F_i depend only on variables in some small subset of V?Suggestions on the presentation:- you use the word "subspace" throughout the paper. This is a very general term, I would suggest to use "linear subspace" to remove potential ambiguities.- in Corollary 8 you could add that this set can be equivalently described by equalities $s(A_i)=0$, then you wouldn't need matrix T'.- In lines 305 and 310 you use aff(A_x). It could be better to use aff_0(A_x), since that's how the convergence results (e.g. Corollary 5) are formulated.I think it is important to state in the abstract which specific method you analyze (with a reference), and your main result, i.e. the convergence rate that you obtain.


Optimistic planning in Markov decision processes using a generative model
We consider the problem of online planning in a Markov decision process with discounted rewards for any given initial state. We consider the PAC sample complexity problem of computing, with probability 1−δ, an �-optimal action using the smallest possible number of calls to the generative model (which provides reward and next-state samples). We design an algorithm, called StOP (for StochasticOptimistic Planning), based on the "optimism in the face of uncertainty" principle. StOP can be used in the general setting, requires only a generative model, and enjoys a complexity bound that only depends on the local structure of the MDP.

The paper considers the problem of online planning in a MDP with discountedrewards. More precisely, the problem is, for any given initial state, to decide an action (at the initial state) that is epsilon-optimal with high probability.The algorithm is assumed to have access to the generative model, i.e.,an oracle that, when invoked with a state-action pair, provides a reward anda next-state according to the probability distributions associated with thetarget MDP. The paper designs an algorithm called StOP with sample complexitynearly as small as the best baseline algorithm OP-MDP, which is assumed tohave access to the true distributions over rewards and next-states. Thus,the advantage of StOP is obvious.The sample complexity of StOP derived depends on a parameter kappa, which isa complexity measure of the target MDP. Intuitively, kappa measures hardnessof approximating the optimal reward using random rollouts in the look-ahead(action-transition) tree.The paper is well-motivated and clearly written. The sample complexity boundderived is good. It seems that the idea behind the algorithm is a reductionto the best-arm identification, which is a standard way of estimating the gamevalue by random playouts in the game tree (c.f. UCT). So it should beclarified what is the technical contribution and what is the key to derivingthe small sample complexity.Some specific comments:It is unclear how the authors derive from Theorem 3 the sample complexityin terms of kappa given in Introduction. Indeed, kappa is not defined inthe body of the paper.It would be helpful to give intuitions about the meanings of kappa and/orS^{epsilon,*}. It is important to characterize (or upperbound) the kappain terms of some simpler and combinatorial measures of MDP, and therebyto make it clear what structures of MDPs make the proposed method effective.


Do Convnets Learn Correspondence?
Convolutional neural nets (convnets) trained from massive labeled datasets [1] have substantially improved the state-of-the-art in image classification [2] and object detection [3]. However, visual understanding requires establishing correspondence on a finer level than object category. Given their large pooling regions and training from whole-image labels, it is not clear that convnets derive their success from an accurate correspondence model which could be used for precise localization. In this paper, we study the effectiveness of convnet activation features for tasks requiring correspondence. We present evidence that convnet features localize at a much finer scale than their receptive field sizes, that they can be used to perform intraclass aligment as well as conventional hand-engineered features, and that they outperform conventional features in keypoint prediction on objects from PASCAL VOC 2011 [4].

The paper analyzes the degree to which features in CNNs identify object or part locations in images, even though they may not be trained for localization. There are three sets of tests. The first is to find nearest neighbors for patches based on their CNN activations; the second warps an image to its nearest neighbor patches, and the third transfers keypoints. The results of these experiments indicate that CNN features could be a replacement for SIFT in computer vision tasks. Overall, the topic and results here generally justify publication of the paper. The paper addresses a very interesting and relevant question that could definitely have impact on the vision literature.There are two major problems with the paper.The first problem is that most of the results are very unconvincing as they are presented. * The results in Figure 1 are not trustworthy. It is easy for a human viewer to be mislead by such a figure, i.e., to infer that correspondences are correct even when they are not, like in a Photomosaic (http://www.photomosaic.com/). More needs to be done to verify that the claimed correctness of correspondences is actually correct, like showing a visualization of class/part labels. Also, it doesn't help that the resolution of this figure is terrible; it is blurry even at print resolution. * I do not understand the results in Figure 2 at all. It is unclear what is meant by receptive field here: I would assume it's the domain of the filters, but pixels are shown here.* The results in Figure 3 are unpersuasive and ambiguous, but the scores in Table 2 seem fine to support the point that the results are comparable to SIFT (though not better). * The results in Figure 6 and Table 4 are persuasive to me, but I'm not an expert in keypoint prediction, so I can't really judge the significance of this result or the experimental setup.Second, the paper is not clearly written; it reads like a class project report, in which many terms and acronyms are used without definition. In many cases one can make educated guesses (and being familiar with the literature and the Caffe implementation helps), but one should not have to guess what is in the authors' heads. Examples: it is never defined how CNNs produce "features" (presumably the vector of filter outputs); the acronym "rfs" is never defined, nor is the receptive field that it refers to; in Section 3, the problem statement and inputs and outputs are never defined, nor is the source or target image. It's not actually clear to me what is meant by receptive field (see above).One other comment: an impact of this paper is that it might encourage vision researchers to replace SIFT with CNN features. It would be helpful to hear about the practicalities of doing so, in particular the memory and computation time requirements of doing so.


General Stochastic Networks for Classification
We extend generative stochastic networks to supervised learning of representations. In particular, we introduce a hybrid training objective considering a generative and discriminative cost function governed by a trade-off parameter λ. We use a new variant of network training involving noise injection, i.e. walkback training, to jointly optimize multiple network layers. Neither additional regularization constraints, such as 1, 2 norms or dropout variants, nor pooling-or convolutional layers were added. Nevertheless, we are able to obtain state-of-the-art performance on the MNIST dataset, without using permutation invariant digits and outperform baseline models on sub-variants of the MNIST and rectangles dataset significantly.

This paper extends the General Stochastic Network (GSN) models to perform simultaneous generative and discriminative tasks, via a hybrid training objective that balances a data reconstruction and label prediction. The model is trained using the recently introduced walkback algorithm, which yields a generative model by first injecting noise and then learning to remove it, and provides an alternative to other regularization techniques. The model is trained on several variants of MNIST, showing improvement with respect to previous methods. The paper has two main contributions. (i) extension of GSNs to perform simultaneous discriminative and generative task, (ii) introduce a hybrid cost function, with a trade-off parameter that controls the balance between discriminative and generative terms and an annealing scheme. I think that each contribution, taken individually, is relatively minor and incremental, but the overall model is interesting, with potential to drive further improvements, and shown to be effective in several tasks. Pros:- Good numerical results: state-of-the-art results on MNIST permutation invariant ( If I understood correctly, the performance of the network on that dataset would have been the same if the pixels were shuffled, hence it is not using the pixel geometry - if that is the case, I do not understand the terminology 'without permutation invariant digits' ). - Interesting alternative to the classical unsupervised pretraining + discriminative fine-tuning paradigm. Although the annealing scheme effectively performs something similar in spirit, the transition is smoother and the framework is easier to study. The reviewer is however not expert enough to say whether this is a novelty. - Potential for improvement: As the authors mention, the model seems robust to hyperparameters (number of walkback steps and noise level) and prone to generalization to more sophisticated tasks. Cons/Remarks:- Somehow the paper mixes two different aspects that might confuse the reader: (i) how to combine generative with discriminative tasks, and (ii) how to obtain efficient generative models. The paper considers the GSN framework, which is a valid choice, but I would have liked to see a more clear comparison (perhaps a section?) where this model is compared with traditional (noise-free) autoencoders which use a reconstruction term and a discriminative term. The authors mention in the numerical experiments section that the GSN model with no noise also performed well in experiments. Is that equivalent to a standard transforming autoencoder enriched with a discriminative cost? - Additive interactions in the Markov chains. Figures 2 and 3 show the basic architecture of the GSN, which generalizes the Denoising Autoencoder Markov chain. In this extended model, hidden random variables are modeled as sums of transition functions coming from different sources. I would have expected the influence of different sources to have a nonlinear effect in the state of hidden variables, such as multiplicative interactions.- Is the Gaussian additive noise (pre and post activation) sufficient to achieve multimodality? Could the authors describe how this mechanism scales to high-dimensional data (with potentially the number of 'modes' scaling exponentially with the dimension). Section 3.1 describes an analysis of the noise sigma, but sigma is not defined previously (I assumed it is the standard deviation of eta_out and eta_in). Is this noise isotropic? why? - Interpretation of section 3.1. The analysis of the role of walkback steps and the noise level is critical for the understanding of the model, and I wished the authors would give further, deeper insight on the results depicted in Figures 6 and 7. Why does the performance depend so little on K? Has the chain already converged? And also, from Figure 7 is seems as if the noise-free version performs nearly as well as the carefully chosen noisy version. Is this because the noise distributions are too simplistic (Isotropic gaussian, with same variance across the network?), or is there a more fundamental reason?