Matrix Completion from Fewer Entries: Spectral Detectability and Rank Estimation
The completion of low rank matrices from few entries is a task with many practical applications. We consider here two aspects of this problem: detectability, i.e. the ability to estimate the rank r reliably from the fewest possible random entries, and performance in achieving small reconstruction error. We propose a spectral algorithm for these two tasks called MaCBetH (for Matrix Completion with the Bethe Hessian). The rank is estimated as the number of negative eigenvalues of the Bethe Hessian matrix, and the corresponding eigenvectors are used as initial condition for the minimization of the discrepancy between the estimated matrix and the revealed entries. We analyze the performance in a random matrix setting using results from the statistical mechanics of the Hopfield neural network, and show in particular that MaCBetH efficiently detects the rank r of a large n × m matrix from C(r)r √ nm entries, where C(r) is a constant close to 1. We also evaluate the corresponding root-mean-square error empirically and show that MaCBetH compares favorably to other existing approaches. Matrix completion is the task of inferring the missing entries of a matrix given a subset of known entries. Typically, this is possible because the matrix to be completed has (at least approximately) low rank r. This problem has witnessed a burst of activity, see e.g. [1,2,3], motivated by many applications such as collaborative filtering [1], quantum tomography [4] in physics, or the analysis of a covariance matrix [1]. A commonly studied model for matrix completion assumes the matrix to be exactly low rank, with the known entries chosen uniformly at random and observed without noise. The most widely considered question in this setting is how many entries need to be revealed such that the matrix can be completed exactly in a computationally efficient way [1,3]. While our present paper assumes the same model, the main questions we investigate are different.
The first question we address is detectability: how many random entries do we need to reveal in order to be able to estimate the rank r reliably. This is motivated by the more generic problem of detecting structure (in our case, low rank) hidden in partially observed data. It is reasonable to expect the existence of a region where exact completion is hard or even impossible yet the rank estimation is tractable. A second question we address is what is the minimum achievable root-mean-square error (RMSE) in estimating the unknown elements of the matrix. In practice, even if exact reconstruction is not possible, having a procedure that provides a very small RMSE might be quite sufficient.
In this paper we propose an algorithm called MaCBetH that gives the best known empirical performance for the two tasks above when the rank r is small. The rank in our algorithm is estimated as the number of negative eigenvalues of an associated Bethe Hessian matrix [5,6], and the corresponding eigenvectors are used as an initial condition for the local optimization of a cost function commonly considered in matrix completion (see e.g. [3]). In particular, in the random matrix setting, we show 1
Matrix completion is the task of inferring the missing entries of a matrix given a subset of known entries. Typically, this is possible because the matrix to be completed has (at least approximately) low rank r. This problem has witnessed a burst of activity, see e.g. [1,2,3], motivated by many applications such as collaborative filtering [1], quantum tomography [4] in physics, or the analysis of a covariance matrix [1]. A commonly studied model for matrix completion assumes the matrix to be exactly low rank, with the known entries chosen uniformly at random and observed without noise. The most widely considered question in this setting is how many entries need to be revealed such that the matrix can be completed exactly in a computationally efficient way [1,3]. While our present paper assumes the same model, the main questions we investigate are different.
The first question we address is detectability: how many random entries do we need to reveal in order to be able to estimate the rank r reliably. This is motivated by the more generic problem of detecting structure (in our case, low rank) hidden in partially observed data. It is reasonable to expect the existence of a region where exact completion is hard or even impossible yet the rank estimation is tractable. A second question we address is what is the minimum achievable root-mean-square error (RMSE) in estimating the unknown elements of the matrix. In practice, even if exact reconstruction is not possible, having a procedure that provides a very small RMSE might be quite sufficient.
In this paper we propose an algorithm called MaCBetH that gives the best known empirical performance for the two tasks above when the rank r is small. The rank in our algorithm is estimated as the number of negative eigenvalues of an associated Bethe Hessian matrix [5,6], and the corresponding eigenvectors are used as an initial condition for the local optimization of a cost function commonly considered in matrix completion (see e.g. [3]). In particular, in the random matrix setting, we show that MaCBetH detects the rank of a large n × m matrix from C(r)r √ nm entries, where C(r) is a small constant, see Fig. 2, and C(r) → 1 as r → ∞. The RMSE is evaluated empirically and, in the regime close to C(r)r √ nm, compares very favorably to existing approache such as OptSpace [3].
This paper is organized as follows. We define the problem and present generally our approach in the context of existing work in Sec. 1. In Sec. 2 we describe our algorithm and motivate its construction via a spectral relaxation of the Hopfield model of neural network. Next, in Sec. 3 we show how the performance of the proposed spectral method can be analyzed using, in parts, results from spin glass theory and phase transitions, and rigorous results on the spectral density of large random matrices. Finally, in Sec. 4 we present numerical simulations that demonstrate the efficiency of MaCBetH. Implementations of our algorithms in the Julia and Matlab programming languages are available at the SPHINX webpage http://www.lps.ens.fr/~krzakala/WASP.html.

This paper proposes a method for estimating the rank of a low-rank incomplete data matrix, based on the so-called Beth free energy, which is an energy function used in statistical mechanics. The authors use the incomplete data matrix to define a graphical model on a bipartite graph, corresponding to the rows and columns of the data matrix. This graphical model is further studied by an approximation called the Beth free energy (BFE), whose phase transition is known in statistical mechanics: as a function of a certain parameter beta, the BFE has a single global optimum for small enough beta. As beta increases a few local minima appear, called the retrieval states. As beta keeps increasing a phase transition occurs in that many spurious local minima appear. The authors use a correspondence between statistical mechanics and the appearence of phase transitions in the graphical model, to estimate the rank of the underlying matrix as the number of retrieval states. This correspondence enables the authors to estimate the minimum number of observed entries per row required to correctly estimate the rank of the matrix. Experiments using synthetic data demonstrate the merit of the proposed method. From reading the paper it is clear that the authors are very knowledgable as far as the Beth approximation is concerned, and their literature review on matrix completion is fair. In addition, their introduction up to the end of page 2 is well written and comprehensive. More importantly, the idea of using the Beth free energy for the purpose of rank-estimation of low-rank incomplete matrices seems original and as the synthetic experiments demonstrate, quite promising. I believe that the right way to judge this paper is as a work that attempts to bring together notions from machine learning (matrix completion) and notions from statistical mechanics (phase transition of Beth free energy). From such a viewpoint the paper fails, as the authors do not clearly establish the correspondence that they are using to translate the physics notions to the machine learning notions. Let me be precise. In the paper, there seems to be a length-2 chain of correspondences: Matrix Completion - Graphical Models - Statistical Mechanics. As far as the first correspondence is concerned, the authors give no motivation or intuition at all at writing down graphical model (6). What does this model mean in terms of matrix completion? Next, from graphical model (6) the authors pass to the Beth free energy (7), for which they claim that its phase transition is well known. But in line 182 they claim without further comment, perhaps the most important claim of this paper, that a good estimate for the rank of the matrix is the number of retrieval states of (7). Where does that come from? I believe the authors should very carefully justify this claim, as this seems to be the backbone of the paper. Next, at lines 194-197 they say that to study the retrieval states they look at the negative eigenvalues of the Hessian of the BFE at its global minimum (paramagnetic state). Again, how is this justified? At lines 266-269 the authors state an important characterization of retrieval states once again without justification. Naturally, i do not expect the authors to reproduce a possibly large amount of results from statistical mechanics so that all arguments are self-contained and rigorous. But i do expect them to tie connections on the high-level, and in the low-level wherever possible, with the problem they are trying to solve, which is estimating the rank and more importantly, they should spend more space on establishing very clearly the correspondences that are being exploited.This paper has the potential of becoming a good paper subject to the authors rewriting it in a more accessible way, and establishing carefully all important connections for their purpose between matrix completion and statistical mechanics.As the paper stands, i can not recommend it for publication, since i believe that many readers would not even understandits basic ideas.


When are Kalman-Filter Restless Bandits Indexable?
We study the restless bandit associated with an extremely simple scalar Kalman filter model in discrete time. Under certain assumptions, we prove that the problem is indexable in the sense that the Whittle index is a non-decreasing function of the relevant belief state. In spite of the long history of this problem, this appears to be the first such proof. We use results about Schur-convexity and mechanical words, which are particular binary strings intimately related to palindromes.

This paper attempts to answer a question of "indexability" for simple Kalman filter, restless bandits. The context used is that of tracking N independent time-series, each of which follows a Gaussian random walk. At each time point (discrete time) an action is made to observe one of these time-series with higher precision than the others. The task is to choose the actions (or a policy producing the actions) which minimises a weighted sum of the error variances. The authors cast this problem as a multi-arm restless bandit and ask if previous work on "indexing" can be applied to it. The main result of the paper is to prove that the bandit is indexable under certain assumptions on the optimal policy.The first part of the paper lays out the problem as has just been described, as well as stating the main theorem of the paper. The authors do this with a reasonable amount of clarity. That said, some sentences could be written more clearly (e.g. the last sentence of the opening paragraph).The bulk of the paper is then made up of a proof of the main theorem. This involves the use of mathematical concepts which I suspect are relatively unknown amongst the community. The authors do a commendable job of trying to introduce all the definitions and lead the reader through the proof.While the result of the paper is interesting I feel the authors could have done more to explain why it is important. Clearly there is a significant limit on space due to the necessity of introducing the mathematical concepts, but there is nothing in the paper to tie the result back to the original context in which the problem was set.


Reflection, Refraction, and Hamiltonian Monte Carlo
Hamiltonian Monte Carlo (HMC) is a successful approach for sampling from continuous densities. However, it has difficulty simulating Hamiltonian dynamics with non-smooth functions, leading to poor performance. This paper is motivated by the behavior of Hamiltonian dynamics in physical systems like optics. We introduce a modification of the Leapfrog discretization of Hamiltonian dynamics on piecewise continuous energies, where intersections of the trajectory with discontinuities are detected, and the momentum is reflected or refracted to compensate for the change in energy. We prove that this method preserves the correct stationary distribution when boundaries are affine. Experiments show that by reducing the number of rejected samples, this method improves on traditional HMC.

SUMMARY        Hamiltonian MCMC methods sample from a probability       distribution by treating its log as a "potential energy"       function over the state space, augmenting the space with extra       "momentum variables" and their associated "kinetic energy",       and evolving the state of the Markov process by integrating       the physical Hamiltonian equations of motion of the system.       Each step of the Markov chain is accomplished by numerically       integrating the Hamiltonian equations forward in time.       However, if the energy function is non-differentiable, the       integral is not well-defined.The rejection step that is used       to counteract numerical inaccuracies in the integration also       accounts for such non-differentiable regions, but at the cost       of slowing down the mixing rate of the Markov chain.This       paper suggests physically-inspired "reflections" and       "refractions" of the trajectory of the system that occur       whenever the state crosses a discontinuity in the energy       function.It applies to target distributions that are       differentiable everywhere except on the boundaries of certain       polytopes; the reflection or refraction occurs whenever the       trajectory of the system crosses such a boundary.        Whether a reflection or refraction occurs depends on if the       momentum component of the state is sufficient to "climb" the       gap in energy.The authors show that the necessary       volume-preservation properties are satisfied by these       reflection and refraction procedures, so that the Markov       chains converges to the required target distribution.        QUALITY        The presented modification to Hamiltonian MCMC is useful and       the case of piecewise-differentiable target distributions is       quite applicable in practice.It would, of course, be useful       to have boundaries that are not affine hyperplanes.        CLARITY        The algorithm, its motivation, and the proof of the volume       preservation property is clearly presented.Some more       (qualitative and/or quantitative) discussion about how the       reflections and refractions affect rejection rates would be       useful.        ORIGINALITY AND SIGNIFICANCE        The idea of applying physically-based reflection and       refraction to Hamiltonian MCMC methods appears to be novel and       useful.While the present work proves its correctness under       certain conditions, it is an interesting avenue of research to       extend it further.


Adaptive Online Learning
We propose a general framework for studying adaptive regret bounds in the online learning setting, subsuming model selection and data-dependent bounds. Given a data-or model-dependent bound we ask, "Does there exist some algorithm achieving this bound?" We show that modifications to recently introduced sequential complexity measures can be used to answer this question by providing sufficient conditions under which adaptive rates can be achieved. In particular each adaptive rate induces a set of so-called offset complexity measures, and obtaining small upper bounds on these quantities is sufficient to demonstrate achievability. A cornerstone of our analysis technique is the use of one-sided tail inequalities to bound suprema of offset random processes. Our framework recovers and improves a wide variety of adaptive bounds including quantile bounds, second order data-dependent bounds, and small loss bounds. In addition we derive a new type of adaptive bound for online linear optimization based on the spectral norm, as well as a new online PAC-Bayes theorem.

Summary:This paper investigates luckiness guarantees for online learning that may depend on the data and on the comparator. Deciding achievability of a proposed adaptive bound is cast as a minimax problem. This problem is subsequently relaxed using sequential Rademacher complexity. Probability tail bounds are used to give tractable expressions for achievable bounds in general and in a variety of examples. Originality and Significance:This is a novel extension of the game-theoretic approach to obtain adaptive bounds. Whereas most algorithms are designed bottom-up, this approach starts at the core of the problem and. Tractability then becomes a trade-off with sharpness of bounds. This method is very powerful, and will undoubtedly result in new algorithms for existing and new adaptivity targets. Clarity and Quality:The paper is nicely done. I was a little disappointed that the methods, after plugging in the proposed tail bounds, unfortunately cannot match the state of the art in the worked-out examples. I also feel that the discussion of the results is a little one-sided as it does not point this out enough.  The bound of Example 4.4 should be compared to Theorem 1 and Corollary 1 from the NIPS'14 paper "Exploiting Easy Data in Online Optimization" by Sani, Neu and Lazaric. Their bound seems strictly better, as it gives constant regret compared to the singled-out strategy, independent of the time horizon and the number of other strategies. I am also a little wary of the current bound's dependence on $L$. Intuitively the whole bound should simply scale with $L$, for this is what should happen if we rescale the loss function. (if not, the results can be improved by pre- and post-scaling appropriately). The bound of Example 4.5 should be compared to Theorem 4 (which trivially extends from subsets to mixtures) from the COLT'15 paper "Second-order Quantile Methods for Experts and Combinatorial Games" by Koolen and Van Erven. Doing so, the advantage that the current paper sees over [13] (which is obtained by magnifying lower-order terms) disappears. What remains is that the current paper has a worse linear term (loss of comparator f instead of variance of regret with respect to f) at the cost of a worse multiplicative overhead (ln n at time horizon n instead of ln ln n). It is surprising that the proposed method, which in principle should be much tighter as it is minimax-optimal, ignores all computational issues and fixed-horizon, loses out on all fronts to a computationally efficient anytime method. It would be interesting to see a dissection of where the advantage is lost.  Minor: line 161: move the left-hand-side of this definition from line 160 into the equation. lines 272, 372: You need a singleton $\cal X$. If it is empty everything collapses. line 297: Why use O-tilde if you spell out the log factor? line 304: It looks that empty $\cal F(1)$ presents problems. For what is the Rademacher complexity of the empty set? If it is $0$ we have a division by $0$. If it is $-\infty$ the bound 304 is unachievable. line 311: Why is infinite set of radii an issue for exponential weights? line 336: omit "is" Example 4.4. I was unsure what $N$ means here. Is $N = |\cal F|$? line 418: Perhaps omit this, it sounds a little embarrassing. You did find the appendix for other material. page 9: please unify the reference style (full names vs initials) and check for capitals in titles: Hilbert, Banach, ...         *** UPDATE *** I would like to respond to this sentence from the author response:The bound of Koolen et al is N-independent, but is in terms of the prior (that is, MDL-style).The suggested difference is purely cosmetic and not intrinsic. A trivial modification of the analysis results in a KL (i.e. PAC-Bayes) bound. Koolen and Van Erven seem to do this for their Theorem 8 but not their Theorem 4. Yet, there is no difficulty whatsoever.


Parallel Multi-Dimensional LSTM, With Application to Fast Biomedical Volumetric Image Segmentation
Convolutional Neural Networks (CNNs) can be shifted across 2D images or 3D videos to segment them. They have a fixed input size and typically perceive only small local contexts of the pixels to be classified as foreground or background. In contrast, Multi-Dimensional Recurrent NNs (MD-RNNs) can perceive the entire spatio-temporal context of each pixel in a few sweeps through all pixels, especially when the RNN is a Long Short-Term Memory (LSTM). Despite these theoretical advantages, however, unlike CNNs, previous MD-LSTM variants were hard to parallelise on GPUs. Here we re-arrange the traditional cuboid order of computations in MD-LSTM in pyramidal fashion. The resulting PyraMiD-LSTM is easy to parallelise, especially for 3D data such as stacks of brain slice images. PyraMiD-LSTM achieved best known pixel-wise brain image segmentation results on MRBrainS13 (and competitive results on EM-ISBI12).

The authors set out to solve a problem that the highly promising architecture of MDLSTM-RNNs have, which is the ability to parallelise computations. A novel sequential flow of information is proposed, which arguably makes parallelisation much easier. Major issues ===i) MDRNNs are not treated fairly. Fig 1 a) and Fig 2 left are dangerously misleading: they make the impression that it takes four MDRNNs to scan a whole 2d plane. But this is not true, the scanning can continue from the center point to the lower right corner, and the prediction can be made there. Consequently, a single sweep of a MDRNN sees *all* the pixels, a theoretical benefit of MDRNNs over Pyramid RNNs, for which this property is not shown.ii) The authors make clear that their computational flow, that of a Pyramid, introduces some independencies which make parallelization easier. What they do not is to elaborate on the consequences of this from a statistical view--i.e. what assumptions are made then.E.g. if we look at Fig 1 c), the pixels which are neighbours along the columns are treated as independent. This seems as a big restriction to me and I think should be discussed in the text--e.g. is it possible to undo this dependency? (see next point)iii) The C-LSTM layer is not justified, the authors do not give reasons why they do it. I can imagine that it helps to make up for the dependencies that the Pyramid layer is lacking--but the authors should make clear why this is done.iv) It is not explained very well how the segmentation is actually done. The paper makes the impression that a single pass over a volume forms a single prediction--the output of the softmax. But what if many pixels are to be segmented? Do the authors propose to sweepover the volume once per segmentation label? Further, why are the authors chosing the MSE as a training criterion, opposed to the negative log-likelihood of the categorical distribution induced by the softmax?v) The experiments are very domain specific, while the method is not. I don't see why no results on more standard benchmarks are presented, e.g. Pascal VOC. This would add a lot of value to the paper, as it eases the reader's effort to evaluate the method for applications in different contexts. Minor issues ===i) I think that Lucas Theis' work on generative modelling with spatial LSTMs [2] might (!) be of interest for the related work section.ii) The LSTM explanation is poor. The x symbol is overloaded, it is not clear what dimensionality the different quantities have, \text{} is not used in the subscripts, also the reader has to guess what x is. It should be easy to improve this to take cognitive burden from the reader.iii) Introduction, "neighbouring" -> "predecessing".iv) The correct cite for dropout on the non-recurrent connections of RNNs is [1], who did that earlier than Google. [1] Pham, Vu, et al. "Dropout improves recurrent neural networks for handwriting recognition." Frontiers in Handwriting Recognition (ICFHR), 2014 14th International Conference on. IEEE, 2014.[2] @unpublished{Theis2015c, author = "L. Theis and M. Bethge", title = "Generative Image Modeling Using Spatial LSTMs", year = 2015, journal = "arXiv", month = "Dec", keywords = "deep learning, generative modeling, natural image statistics, lstm, mcgsm", url = "http://arxiv.org/abs/1506.03478/" }


Convolutional Networks on Graphs for Learning Molecular Fingerprints
We introduce a convolutional neural network that operates directly on graphs. These networks allow end-to-end learning of prediction pipelines whose inputs are graphs of arbitrary size and shape. The architecture we present generalizes standard molecular feature extraction methods based on circular fingerprints. We show that these data-driven features are more interpretable, and have better predictive performance on a variety of tasks.

This paper describes a method for learning vector representations of small molecules, or 'fingerprints,' using deep, convolutional neural networks.This is high-quality work, and makes a significant contribution to the important problem of learning useful representations for chemoinformatics. It is clearly written and a pleasure to read.In terms of originality, the neural network architecture presented here is pretty similar to the work of Lusci et al 2013. The three differences in network topology are: 1) This paper proposes the use of a different set of NN weights at each layer of the network, while Lusci ties the weights in each layer together, making the network recursive. 2) This method avoids cycles by limiting the depth of the network, compared to the approach in Lusci et al where the recursive structure rooted at each atom is forced to be a tree (cycles are broken).3) Here, there are intermediate connections from each layer of the recursive network to the output.But these differences, along with other technical choices (e.g. the use of the softmax activation in the output layer), can be important. The technical challenges of using methods like this can be prohibitive to people working in chemoinformatics, so relating this algorithm to the familiar technique of circular fingerprinting is valuable. There are plenty of original perspectives, ideas, and experiments in this work to warrant publication.Specific comments:In Figure 2, there should be a comma in line 8 between ra and r1. Also, atoms have a variable number of neighbors, and the text should mention how this is handled.The circular fingerprints used in the paper have 2048 binary features, which have the advantage of taking up much less memory than 2048 floats. While this doesn't detract from the importance of the supervised learning results presented here, it could be a limitation for doing things like similarity searches over millions or billions of chemicals.The size of the training and test sets were not clear. Were there 1000 training examples from each data set? 1000 minibatches of size 100?The hyperparameters chosen by spearmint should be given, preferably in a table along with the bounds of the search space.One major difficulty with architectures like this is that the computational benefits of doing SGD with minibatches are often lost because the NN architecture for each molecule is different. How was minibatch training done here?


A Framework for Individualizing Predictions of Disease Trajectories by Exploiting Multi-Resolution Structure
For many complex diseases, there is a wide variety of ways in which an individual can manifest the disease. The challenge of personalized medicine is to develop tools that can accurately predict the trajectory of an individual's disease, which can in turn enable clinicians to optimize treatments. We represent an individual's disease trajectory as a continuous-valued continuous-time function describing the severity of the disease over time. We propose a hierarchical latent variable model that individualizes predictions of disease trajectories. This model shares statistical strength across observations at different resolutions-the population, subpopulation and the individual level. We describe an algorithm for learning population and subpopulation parameters offline, and an online procedure for dynamically learning individual-specific parameters. Finally, we validate our model on the task of predicting the course of interstitial lung disease, a leading cause of death among patients with the autoimmune disease scleroderma. We compare our approach against state-of-the-art and demonstrate significant improvements in predictive accuracy.

The authors design and fit a hierarchical Bayesian model for predicting disease trajectories (i.e., a scalar measure of disease severity measured throughout the course of the disease) for individual patients. The overall model is an additive combination of a a number of terms including: (1) a population-level term, (2) a subpopulation term, (3) an individual term, (4) a GP term for structured errors. Each of these terms is a function of time, which is modeled parametrically in terms of the coefficients on pre-defined basis expansions (linear and/or B-splines). The subpopulation term involves a discrete mixture model, and the individual level term is a Bayesian linear regression. Distributions are chosen to be Gaussian, which makes most steps of inference and learning work out nicely. The discrete mixture leads to a GMM-like EM algorithm. The authors tune hyperparameters by hand, use likelihood-based learning (EM) for regression coefficients, and use posterior inference to predict the disease trajectory as it evolves. The qualitative results suggest that the method is effective at picking up the signal in lung-disease trajectories to make accurate predictions. The quantitative results demonstrate significant improvements over reasonable baseline. Quality:I find this to be a well done application paper. The model is not earth shattering, but not trivial either. It is clearly explained and appropriate to the problem at hand. The inference and learning methods are carefully derived. The experiments are designed well and demonstrate the value of the method. Note: for the qualitative results, please comment on the range of different results. Are the examples in Figure 2 typical, or were these chosen as particularly compelling examples? Clarity:See above. The paper is well written, and the methods are explained clearly. One small comment: the related work section was hard to understand without having first seeing some details of the proposed method---e.g., to understand why plain GPs would not do the job.Originality:To the best of my knowledge, the model structure proposed here is original. It reuses fairly standard components, so it is not highly original. Because most of the components of the model are Gaussian, the math generally works out nicely. But there is enough complexity in the model components (splines, discrete mixture model, Bayesian linear regression, GP) that the careful derivation of inference and learning updates represent a contribution. When that is all done, standard algorithms can solve the inference and learning problems. Significance:The primary contributions are in the application domain area (health modeling): the design of a new model, derivation of inference and learning algorithms, and demonstration that the method works on a real task. The model could potentially apply elsewhere, but there is not a convincing case made here of significance/relevance as a general-purpose model.


Fast Rates for Exp-concave Empirical Risk Minimization
We consider Empirical Risk Minimization (ERM) in the context of stochastic optimization with exp-concave and smooth losses-a general optimization framework that captures several important learning problems including linear and logistic regression, learning SVMs with the squared hinge-loss, portfolio selection and more. In this setting, we establish the first evidence that ERM is able to attain fast generalization rates, and show that the expected loss of the ERM solution in d dimensions converges to the optimal expected loss in a rate of d/n. This rate matches existing lower bounds up to constants and improves by a log n factor upon the state-of-the-art, which is only known to be attained by an online-to-batch conversion of computationally expensive online algorithms.

SUMMARY: Studying Empirical Risk Minimization (ERM) with exp-concave loss, which is more general than strong convexity, is the topic of this paper. The goal is to provide an upper bound for the excess loss. The domain is assumed to be a compact subset of R^d. In addition to the exp-concavity, the smoothness of the loss function is assumed too. Moreover, the estimator requires a small amount of strongly convex regularization to be added to the ERM objective. The paper shows that under these assumptions and with the modified estimator, the expectation of the excess loss behaves as O(d/n).The proof has some key elements:Even though we do not have strong convexity in the whole domain, we can still define a local notion of strong convexity, which holds around the empirical minimizer w.r.t. a norm that depends on each of n data points. In other words, the proof does not use a single norm for all points. This leads to an upper bound for leave-one-out kind of stability terms (Lemma 6). Even though the upper bound of Lemma 6 is not uniformly small for all data points, Lemma 8 shows that not many of them can be large (order of d), and the rest are small. These along the i.i.d. property of data allows one to provide a fast upper bound on the expectation of the excess loss.  EVALUATION and QUESTIONS: I think this is a nice theoretical paper. It generalizes the results that are known for strongly convex losses to a more relaxed class of losses. The proofs seem to be correct.Here are some comments and questions:- It might be helpful to provide some concrete examples of exp-concave losses that are used in practice, which are not strongly convex.- What would happen if the parameter domain is not compact? Presumably one cannot prove an upper bound for the unregularized ERM. What about this regularized variant? Is the current 1/n R(w) regularization enough to prove something meaningful?- Is it possible to relax the requirement of the strong convexity of the regularizer? It appears that the proof of Lemma 6 breaks down without that.


A Theory of Decision Making Under Dynamic Context
The dynamics of simple decisions are well understood and modeled as a class of random walk models [e.g. 1-4]. However, most real-life decisions include a dynamically-changing influence of additional information we call context. In this work, we describe a computational theory of decision making under dynamically shifting context. We show how the model generalizes the dominant existing model of fixed-context decision making [2] and can be built up from a weighted combination of fixed-context decisions evolving simultaneously. We also show how the model generalizes recent work on the control of attention in the Flanker task [5]. Finally, we show how the model recovers qualitative data patterns in another task of longstanding psychological interest, the AX Continuous Performance Test [6], using the same model parameters.

The authors extend the popular sequential probability ration test for decisions under perceptual uncertainty to situations in which a there are additional contextual stimuli. This is achieved by simply accumulating samples and computing a posterior distribution over both context and target from which the decision is being generated when a bound is being reached. As the onset and offset of the cues can be modeled separately, the authors decided to give examples for the cases in which target and context appear at the same time and where the target cue appears with a delay after the disappearance of the contextual cue, corresponding to the Eriksen flanker task and the continuous performance test.Finally, the authors set the parameters for the flanker task and use these same parameters for the second task. The behavioral performance is matched well and the corresponding reaction times are within 40% of the empirically measured values of the experiment.This is overall an interesting topic for the cognitive science and neuroscience community that follows NIPS and it is an extension of the sequential probability ration test, which has been extremely popular in the last years. On the other hand, there is other recent work emphasizing the importance of modeling contextual effects in explain human decision making behavior: the authors should relate their approach to previous work in decision-making with contexts e.g. Srivastava & Schrater's work, at least cite it. The current model assumes a perfect separation of the evidence provided for target and context given the sensory stimuli, which is potentially not very realistic. I would call the proposed model a 'model' and not a 'theory' because a model can certainly generate or accommodate multiple experimental settings, but this may be a matter of taste.


Data Generation as Sequential Decision Making
We connect a broad class of generative models through their shared reliance on sequential decision making. Motivated by this view, we develop extensions to an existing model, and then explore the idea further in the context of data imputation -perhaps the simplest setting in which to investigate the relation between unconditional and conditional generative modelling. We formulate data imputation as an MDP and develop models capable of representing effective policies for it. We construct the models using neural networks and train them using a form of guided policy search [9]. Our models generate predictions through an iterative process of feedback and refinement. We show that this approach can learn effective policies for imputation problems of varying difficulty and across multiple datasets.

This paper connects methods for policy search in reinforcement learning with inference techniques for general generative models. Several inference methods are reviewed and extended. The second part of the paper applies these to imputation, which amounts to reconstructing damaged images from e.g. the MNIST dataset.This paper may seem very solid from a technical point of view, but at the same time it is not easy to read "story-wise". I can follow the technical steps in the paper (although I am not an expert in some aspects of the specific models), but for me many of these are described without enough "story". The introduction (and abstract) on the first page are very promising, but then the next five pages are very technical. This can be good, only if provided with enough intuition, such as is done in lines 207etc and 299-301, but there are too few of such fragments. Some of the insights (such as lines 125-126) are the key components of this paper, but they are motivated typically by looking at the formulas describing them. Problem statements such as equation 2 can be the right level of looking at similarities between problems (and solutions) but much of the remaining parts of (e.g.) sections in 2 are too detailed, IMHO. Then again some parts are very specific about upgrading specific techniques (such as 224-231) and performance comparisons, or about representations of policies (as two-layer networks with specific properties), although the paper (as described in the introduction) would be about sequential decision making for generative models (e.g. sampling).What I basically miss is a roadmap starting from the conceptual start of identifying where policy search and general inference meet, all the way through the various models and extensions. Note that I do appreciate the paper being very technical, and very specific/precise, but for me it does not seem always at the right level. The core aspect of the paper (data generation as sequential decision making) is not very novel, since this topic has been approached before (e.g. grammars and reinforcement learning based training of them, to name one example) but the solid way this paper approaches it (in the current state-of-the-art inference techniques) seems a good way to go. The second part of the paper (section 3) does a better job by first describing an intuitive model of imputation (the problem) and then providing technical details of the solution. (I would like to see some more intuition and description of the reward component though, since that may not be obvious to all readers, including me). I might even have the feeling that imputation would be a good problem to start the paper with, as an illustration of the proposed framework. Sections 3.1. and 3.2. are then somewhat unbalanced again in which details they describe and which should be "known" to the reader from existing literature. I'd rather read more intuition again (such as 299-302), and slightly less details. The details concerning the read and write operations in section 3.2. are not well enough described to understand/appreciate what happens here.Overall, the methods and the imputation setting are relatively clear when arriving in section 4. However, the introduction mentions that there are no alternatives to compare to, such that the authors compare internally, and against a baseline. I do not fully understand why one cannot find alternative techniques in the area of computer vision (for example Francis Bach's work, but there are others) when it comes to image reconstruction (in probabilistic/generative settings). The results seem "ok", but there's not much help to evaluate the findings. The authors state that "appear quite promising", and visually inspecting the results may verify that, but again this evaluation is not totally conclusive. At the end of section 4 it seems that performance on this task is important, whereas before it seemed that comparing the extension (of the LSTM model) against the original was important, whereas even earlier it seemed that connecting generative models and sequential decision making was important. Maybe these different goals, with different levels of description, are the weaknesses of the paper. Overall, a solid technical approach with a description that is maybe not at the right level (of detail). The writing is clear, polished, and very precise. In addition, the experimental results are nice, and they show the viability of the approach, but they may not necessarily demonstrate how to pick which method here, and they do not necessarily exemplify the main thesis of the paper (see the title). I guess that it would be much better to transform parts of this paper to focus more on ideas and to provide a heavy technical report with all the details concerning specific Bernoulli distributions and all that. Nevertheless, even in this form it is an interesting paper.


SubmodBoxes: Near-Optimal Search for a Set of Diverse Object Proposals
This paper formulates the search for a set of bounding boxes (as needed in object proposal generation) as a monotone submodular maximization problem over the space of all possible bounding boxes in an image. Since the number of possible bounding boxes in an image is very large O(#pixels 2 ), even a single linear scan to perform the greedy augmentation for submodular maximization is intractable. Thus, we formulate the greedy augmentation step as a Branch-and-Bound scheme. In order to speed up repeated application of B&B, we propose a novel generalization of Minoux's 'lazy greedy' algorithm to the B&B tree. Theoretically, our proposed formulation provides a new understanding to the problem, and contains classic heuristic approaches such as Sliding Window+Non-Maximal Suppression (NMS) and and Efficient Subwindow Search (ESS) as special cases. Empirically, we show that our approach leads to a state-of-art performance on object proposal generation via a novel diversity measure.

SUMMARYThe paper introduces a principled approach to generate a diverse set of relevant object proposals. This is formulated as optimizing a function F defined on sets of bounding boxes. This function is defined as a sum of a relevance term (which encourages individual bounding boxes to be likely to contain an object) and a diversity term. For optimization they commit to a greedy approach which iteratively adds the most promising bounding box, until the desired number of proposals was generated. The greedy procedure was motivated by existing theoretical guarantees on the obtained solution when F is submodular and monotone. Even under this greedy procedure, the computational cost would be prohibitive when using the naive approach of scanning all possible bounding boxes in order to find the best one. To this end, a Branch and Bound (B&B) procedure is employed. Furthermore, the efficiency of the B&B procedure is improved by sharing information between successive steps of the greedy procedure, generalizing ideas from Minoux's Lazy Greedy procedure.COMMENTSOn the positive side, the paper introduces an extensible framework for object proposal generation, which is shown to encompass some previous heuristics like NMS and the one used in the ESS [22] method. The branch and bound procedure is interesting and makes it feasible to (greedily) search for the solution despite the very large number of candidate bounding boxes considered at each step. This involved constructing an upper bound for the marginal gain of F given a set of candidate bounding boxes. One technical novelty consists of generalizing Minoux's Lazy Greedy procedure in order to leverage information from the B&B tree constructed in the previous step of the greedy procedure. This procedure was shown to perform well in practice. Furthermore, the paper is clearly written and well organized.There appears to be an issue with the monotonicity and submodularity proofs presented in the supplementary material. In brief, this is related to the fact that these concepts are defined on functions over sets (not over lists), and the theoretical guarantees mentioned refer to this situation. The proposed method is defined in terms of a list (lines 179-181), but the diversity function should be defined on the corresponding unordered sets (not lists) in order for the theoretical guarantees to be available. However, the definition in eqs. 7a, 7b can lead to different values of the diversity function D, for two lists which differ only in the order of the proposals (e.g. for different orderings, the same proposal might be attached to different reference bounding boxes, leading to different values when summing up the contributions from all proposals) -- therefore the elements of the underlying set cannot be considered in an arbitrary order when evaluating D, so the proof has to be adjusted. How do these aspects influence the theoretical claims regarding the performance of the greedy procedure?The results in figure 4 show a comparison between the proposed method and variants of Edge Boxes and Selective Search. It would be best to compare to additional state-of-the-art methods, such as MCG and CPMC (which are present in table 1). The results in table 1 are obtained under the regime of a small number of proposals (200 per image, but methods like MCG and CPMC can be tuned for as many as thousands of proposals if not more), which is not frequently used in practice. For such reasons, the impact of the proposed method is not clear from an empirical perspective.The extension of the greedy procedure was shown in fig. 6 to lead to evaluating significantly fewer nodes of the B&B graph. It would be interesting to also show the effect in terms of execution time.Minor comments: there is a double caption for fig. 3.


Column Selection via Adaptive Sampling
Selecting a good column (or row) subset of massive data matrices has found many applications in data analysis and machine learning. We propose a new adaptive sampling algorithm that can be used to improve any relative-error column selection algorithm. Our algorithm delivers a tighter theoretical bound on the approximation error which we also demonstrate empirically using two well known relative-error column subset selection algorithms. Our experimental results on synthetic and real-world data show that our algorithm outperforms non-adaptive sampling as well as prior adaptive sampling approaches.

The paper studies the column subset selection problem (CSSP) via adaptive sampling. The mains points are: 1) The adaptive sampling strategy developed in [8] achieves better reconstruction error than continued sampling for additive error CSSP-algorithms, but gives a worse reconstruction error guarantee than continued sampling for relative error CSSP-algorithms.2) The paper proposes a novel adaptive sampling strategy which is a subtle modification to the strategy presented in [8]. Instead of subtracting the full projection for computing the residual matrix, the proposed strategy in the l-th sampling round subtracts the best rank-lk approximation to the target matrix in the span of the columns selected from the first l-1 rounds. The theorem shows that the proposed strategy is able to achieve better theoretical guarantee over continued sampling for relative-error CSSP algorithms.3) The authors implemented the proposed strategy using two relative-error column selection algorithms. Experiments show that they outperform continued sampling and prior adaptive sampling approaches. Overall, I like this paper. It is well written and is a pleasure to read. Although the approach introduced in this paper is incremental to the existing literature, it is solid and technically sound. To me, this paper asks an interesting question and addresses it in a simple but elegant way. I think it should be of interest to a subset of the NIPS audience.On the negative side, the claim that data matrices have rapidly decaying singular values is not always true. For example, Bai-Yin's law points out that if the entries of a m*n (m >= n) matrix A are independent standard normal random variables, then almost surely the largest singular value is around sqrt(m)+sqrt(n), while the smallest singular value is around sqrt(m)-sqrt(n). This suggests that all the singular values of A are pretty similar if A is a tall-and-skinny matrix. Since the theoretical guarantee of this paper heavily depends on the rapid decay of the spectrum of the target matrix, I am afraid that the proposed approach will perform poorly when dealing with matrix with almost flat singular values. I would have liked to see some discussions about this issue.Minor issues: - Line 74: is t rounds -> in t rounds - Line 411: in[15] -> in [15] - In page 7, the font size in figures are not consistent.


Training Restricted Boltzmann Machines via the Thouless-Anderson-Palmer Free Energy
Restricted Boltzmann machines are undirected neural networks which have been shown to be effective in many applications, including serving as initializations for training deep multi-layer neural networks. One of the main reasons for their success is the existence of efficient and practical stochastic algorithms, such as contrastive divergence, for unsupervised training. We propose an alternative deterministic iterative procedure based on an improved mean field method from statistical physics known as the Thouless-Anderson-Palmer approach. We demonstrate that our algorithm provides performance equal to, and sometimes superior to, persistent contrastive divergence, while also providing a clear and easy to evaluate objective function. We believe that this strategy can be easily generalized to other models as well as to more accurate higher-order approximations, paving the way for systematic improvements in training Boltzmann machines with hidden units.

Detailed comments: 40 - You might be interested in [MacKay, David. "Failures of the one-step learning algorithm." 2001.]42 - PCD is a rediscovery of a method that has recurred several times before, and which I believe actually has some nice convergence properties, at least in terms of some limit in the ratio of learning rate and mixing time. [Neal, R. M. (1992). Connectionist learning of belief networks. Artificial Intelligence] is one earlier example of its use that I'm familiar with. A paper by Salakhutdinov suggests that [H. Robbins and S. Monro. A stochastic approximation method. Ann. Math. Stat., 22:400-407, 1951.] is the earliest reference, though I have not read this older paper.48 - Though they tend to break down around criticality, which is exactly where many interesting real world datasets are expected to be. e.g. see mean field theory with TAP doing worse than the random parameter initialization in [Sohl-Dickstein, Jascha, Peter Battaglino, and Michael R. DeWeese. "Minimum probability flow learning." 2011.]72 - nit - common convention is lowercase for vectors, upper case for matrices. maybe w_ij -> W_ij97 - since you never have to sample the visible units, you could us the more flexible sRBMs, rather than RBMs (sRBM = semi-restricted Boltzmann machine, an RBM w/ the addition of visible to visible couplings) (updated -- I see later that when you take your expansion to third order, this would make the sRBM expensive)132 - "where the dependence on beta of the product ..." huh? This didn't make sense to me.194 - re: careful application, what makes this part tricky?202 - "to gradients" -> "the gradients"210 - i.e. implicit differentiation254 - Why do you need multiple points? Doesn't m have a single fixed point?262-267 - Confused by this paragraph.Figure 1 - Label units on the y-axis! right pane - Should note that this pane is evaluating using the same approx. objective as used in training, so not a fair comparison for CD.296 - re: all parameters of training being set identically - this is not fair! For instance, the training gradient in PCD/CD will have a higher variance, and so it will probably perform best w/ a smaller learning rate than EMF. Might consider using a tool like Spearmint [Snoek, Jasper, Hugo Larochelle, and Ryan P. Adams. "Practical Bayesian optimization of machine learning algorithms." Advances in neural information processing systems. 2012.] as a black box (and thus parsimonious) way to set hyperparameters.Figure 2 - These look grayscale not binary. Are they actually samples or are they probabilities of activation? Make sure to label correctly. Samples give a more clear view of performance. - Curious to see P-TAP3!371-377 - What is the computational cost of these algorithms? What is the wall clock time relative to PCD?391 - "over-fitting" suggests that it's getting better on the training data. It's probably not.(one more comment after reading review from Assigned_Reviewer_4 -- score matching would not apply to this specific case, with RBMs with binary visible units. There are a number of other parameter estimation techniques for energy based models that would however -- minimum probability flow, ratio matching, noise contrastive estimation, minimum KL contraction.)


Max-Margin Majority Voting for Learning from Crowds
Learning-from-crowds aims to design proper aggregation strategies to infer the unknown true labels from the noisy labels provided by ordinary web workers. This paper presents max-margin majority voting (M 3 V) to improve the discriminative ability of majority voting and further presents a Bayesian generalization to incorporate the flexibility of generative methods on modeling noisy observations with worker confusion matrices. We formulate the joint learning as a regularized Bayesian inference problem, where the posterior regularization is derived by maximizing the margin between the aggregated score of a potential true label and that of any alternative label. Our Bayesian model naturally covers the Dawid-Skene estimator and M 3 V. Empirical results demonstrate that our methods are competitive, often achieving better results than state-of-the-art estimators.

This paper formulates learning from crowds as a max-margin learning problem. To further consider the uncertainty of the model, it provides two Bayesian solutions: a Bayesian mean estimator and a Gibbs estimator to the posterior regularization term induced from the hinge-loss term in the original max-margin learning problem. Overall, the paper demonstrates a new successful application within the "Regularized Bayesian" framework. The discriminative function for each task and each label was defined and the resulting margin was characterized. The resulting CrowdSVM problem differs from multi-task SVM, in the aspect that multi-class SVM handles individual features, while CrowdSVM represent a single task as dependent points lies on the axises. To consider generative perspective, the regularized Bayesian framework was introduced. The inference techniques (both variational inference and Gibbs sampling based on data augmentation for SVMs) were borrowed from previous RegBayes papers. I have some concerns in the experiments. Since the entropy-based methods are the most competitive ones with max-margin methods, it worth further comparing these two class of methods. The online learning results seem weird at the first sight. To my understanding, the workers were fixed and the tasks comes in a stream. Assuming the tasks are distributed in some regular patterns, then the online result should recover the batch result. However, in the experiments small batch sizes perform significantly worse. Why? The paper did not give a clear answer. Figure 5 in the supplementary material shows some fluctuation in performance, which can be smoothed by repeating experiments.Although having some concern in the experiments, the paper provides a novel max-margin learning strategy for learning from crowd. It is another empirically successful application for Bayesian discriminative learning.


Collaborative Filtering with Graph Information: Consistency and Scalable Methods
Low rank matrix completion plays a fundamental role in collaborative filtering applications, the key idea being that the variables lie in a smaller subspace than the ambient space. Often, additional information about the variables is known, and it is reasonable to assume that incorporating this information will lead to better predictions. We tackle the problem of matrix completion when pairwise relationships among variables are known, via a graph. We formulate and derive a highly efficient, conjugate gradient based alternating minimization scheme that solves optimizations with over 55 million observations up to 2 orders of magnitude faster than state-of-the-art (stochastic) gradient-descent based methods. On the theoretical front, we show that such methods generalize weighted nuclear norm formulations, and derive statistical consistency guarantees. We validate our results on both real and synthetic datasets.

Paper 1260 tackles the problem of matrix completion with side information represented as a graph. The authors claim that this is a general framework in that the side information can come from many different kinds of sources. Major contributions of this paper are scalable learning algorithm for matrix completition with graph structures, connecting it to existing works in matrix factorization literature, and verifying the proposed method theoretically as well as experimentally.As the authors claim, this problem is widely applicable such as recommendation systems with social network information. Considering the size of such dataset, it is important to bulid a scalable algorithm as the authors proposed.This paper is well-structured and well-written. It is good to have both theoretical analysis of consistency and experiments with real data. Also, it is useful to connect the proposed method in the context of wider convex optimization and noisy matrix approximation.Regarding the experiment (Section 6): some baselines in Table 1 are too simple; no one might be interested in comparison with Global/User/Movie mean. I suggest comparing against more recent, state-of-the-art methods such asSalakhutdinov and Mnih, Probabilistic Matrix Factorization [NIPS 2008], Mackey et al., Divide-and-Conquer Matrix Factorization [NIPS 2011], Lee et al., Local Low-Rank Matrix Approximation [ICML 2013].It is impressive that the proposed method runs in O(10^3) seconds with dataset in Table 2. However, to claim scalability, I recommend running the algorithm on larger dataset, such as Netflix or Yahoo Music. This way it is also possible comparing RMSE score with other methods as well.Minor comments: 1) Why Figure 1 and Figure 2 have different format of x-axis? I recommend using format of Figure 2 (10^1, 10^2, ...) instead of plotting log(time) directly. Log-scale without specifying base is ambiguous. 2) Citations are not in NIPS format.


Super-Resolution Off the Grid
Super-resolution is the problem of recovering a superposition of point sources using bandlimited measurements, which may be corrupted with noise. This signal processing problem arises in numerous imaging problems, ranging from astronomy to biology to spectroscopy, where it is common to take (coarse) Fourier measurements of an object. Of particular interest is in obtaining estimation procedures which are robust to noise, with the following desirable statistical and computational properties: we seek to use coarse Fourier measurements (bounded by some cutoff frequency); we hope to take a (quantifiably) small number of measurements; we desire our algorithm to run quickly. Suppose we have k point sources in d dimensions, where the points are separated by at least from each other (in Euclidean distance). This work provides an algorithm with the following favorable guarantees:
• The algorithm uses Fourier measurements, whose frequencies are bounded by O(1/ ) (up to log factors). Previous algorithms require a cutoff frequency which may be as large as ⌦(
• The number of measurements taken by and the computational complexity of our algorithm are bounded by a polynomial in both the number of points k and the dimension d, with no dependence on the separation . In contrast, previous algorithms depended inverse polynomially on the minimal separation and exponentially on the dimension for both of these quantities.
Our estimation procedure itself is simple: we take random bandlimited measurements (as opposed to taking an exponential number of measurements on the hypergrid). Furthermore, our analysis and algorithm are elementary (based on concentration bounds for sampling and the singular value decomposition).

The paper analyzes the problem of super-resolving a collection of spikes from low-pass measurements. It introduces an algebraic approach based on tensor factorization. For k spikes, suggests taking about k log k random locations in Fourier space, plus d deterministic locations, and then sampling on a grid of size (k log k + d)^3 corresponding to all ways of summing three of these locations. Because of the algebraic properties of Fourier measurements, the resulting 3-tensor is of rank k (up to noise). The paper proves that with high probability in the sample locations, the matrix of factors is well-conditioned, and hence the factors can be stably recovered using linear algebraic manipulations. These results are stable under small deterministic noise. An attractive aspect of the approach is that the requirements do not depend exponentially on the signal dimension. There are two main downsides. The first is that the approach is suboptimal from the perspective of compressed sensing - it intrinsically requires k^3 measurements to recover k spikes. The second is that the stability result is limited to very small noise. This may be an artifact of the analysis, or may be intrinsic to this approach. It is somewhat difficult to judge, because numerical results are not provided. Comments in the response would be appreciated. The main (clever) idea here is the reduction of the problem to 3-tensor decomposition; after this, the analysis follows in a natural way. The submission does not provide much basis for evaluating the practicality of the approach. However, from a theoretical perspective it obtains scalings (esp. in higher dimensions, where the number of samples required does not depend exponentially on d) that to my knowledge were not available using previous approaches.  Minor commentsIn the comparison with the work of Candes and Fernandez-Granda [3], it should be mentioned that [3] allows general complex-valued amplitudes. The setup of this submission restricts the amplitudes to lie between zero and one. There are results of Candes and Morgenshtern on the arxiv for this setup, which might be a more relevant point of comparison. The analysis of noisy tensor decomposition forces the noise level to be very small -of \ell^\infty norm about 1/k^4. An advantage is that the noise can be arbitrary. Can the authors comment: (a) is this restriction an artifact of the tensor decomposition algorithm adopted here and its analysis, or an intrinsic limitation to the proposed approach, and (b) can a better stability result be obtained with stochastic (say, Gaussian) noise? The submission appears to have been written in a hurry. Please spellcheck. Also, "One useful fact..." [line 060] is false. I understand what is intended, but that kind of a typo that early on doesn't build confidence. It most applications that this reviewer is aware of, d is a small constant, and so measuring \Delta in L^2 vs L^\infty does not make a substantial difference.


Learning Theory and Algorithms for Forecasting Non-Stationary Time Series
We present data-dependent learning bounds for the general scenario of nonstationary non-mixing stochastic processes. Our learning guarantees are expressed in terms of a data-dependent measure of sequential complexity and a discrepancy measure that can be estimated from data under some mild assumptions. We use our learning bounds to devise new algorithms for non-stationary time series forecasting for which we report some preliminary experimental results.

Summary:This paper studies learning with time series with quite relaxed assumptions on the stochastic process defining the time series: No mixing or stationarity assumptions is required.This is achieved by the use of two concepts:1) Sequential covering number, which roughly speaking is an extension of the standard covering number to a norm defined over binary tree structures, and 2) Discrepancy, which is a measure of the non-stationarity of the process.The paper provides a high probability deviation inequality that is a function of the covering number and discrepancy. It suggests a method for estimating the discrepancy, under some extra assumptions. It also introduces some algorithms for forecasting using the obtained upper bound.Even though having an upper bound that is valid for non-stationary and non-mixing processes might seem almost magical, one might say that the trick is that one can move all the apparent complexity of the process into the discrepancy and the sequential covering number. If the process is too difficult to learn, these quantities would be too large.The interesting point here, however, is that the quantities that measure the complexity of learning are defined at the right level. To understand my comment, notice that the usual approach to study learning for time series is that we start by analyzing the difficulty of stochastic process (through concepts such as mixing) and then under certain assumptions on the stochastic process, we convert it to the usual i.i.d. setting using techniques such as independent blocks. One can then define covering number/Rademacher complexity/etc. over these independent blocks, and use the standard tools that work for independent samples. This approach separates the study of complexity of the stochastic process and the complexity of function/hypothesis space, and may lead to pessimistic results.What this paper does is that it defines the complexity at the joint stochastic process-function space level. This can potentially lead to much tighter results.The key elements of this approach are not necessarily new.Sequential complexities have been defined by Rakhlin, Sridharan, Tewari (NIPS 2010, NIPS 2011, JMLR 2015). A similar discrepancy measure has also been defined and used by Kuznetsov and Mohri (ALT 2014), Mohri and Munoz Medina (ALT 2012), and even before that in the domain adaptation literature.The use of these together, however, is novel as far as I know.  Evaluation:The topic of the paper is interesting and important, the approach is elegant and as far I can tell novel, and the paper itself is more or less well-written. I have some comments below and I encourage the authors to answer and/or consider them. * L142: Is q_t missing in the LHS?* Is optimizing only the discrepancy term of the upper bound of Corollary 3 the right thing to do?Corollary 2 shows that the upper bound depends on the choice of q. In the beginning of Section 4, it is suggested that one should minimize the discrepancy as a function of q and using it to weigh the training samples.If I understand correctly, the covering number also depends on q (as the cover is defined w.r.t. q-weighted l_1 norm). If that is the case, shouldn't it be considered in the optimization too? (I understand that this might be computationally very difficult or impossible.)* Please compare your algorithms with those in Kuznetsov and Mohri, "Forecasting Non-Stationary Time Series: From Theory to Algorithms," NIPS 2014 Workshop on Transfer and Multi-task Learning. Montreal, Canada, December 2014.* Is it possible to have Bernstein-like bound in Theorem 1?* L222: It seems that the first inequality can actually be an equality.* L246: It might be more clear to use c_t(z_t(\sigma)) instead of c_t(\sigma).* L251: exp(lambda alpha) -> exp( 2 lambda alpha)* L256: c_T(c) -> c_t(sigma)* Some examples of the behaviour of E[N_1(alpha,G,v)] would be helpful. For example, if the process has a certain mixing property, is it different from when it is i.i.d.?* L137: "definition definition" -> "definition"* L177: "leg" -> "lag"* L339-342: The discussion is a bit vague here.* L350: delta' -> delta* L492 (Appendix A): Maybe it is more clear to say that (z,z') is from T(p).* L598: Isn't the change from q to the uniform u too loose? This leads to M || q - u ||_1, which kind of ignores all the structure in the function space except its boundedness.


Bidirectional Recurrent Convolutional Networks for Multi-Frame Super-Resolution
Super resolving a low-resolution video is usually handled by either single-image super-resolution (SR) or multi-frame SR. Single-Image SR deals with each video frame independently, and ignores intrinsic temporal dependency of video frames which actually plays a very important role in video super-resolution. Multi-Frame SR generally extracts motion information, e.g., optical flow, to model the temporal dependency, which often shows high computational cost. Considering that recurrent neural networks (RNNs) can model long-term contextual information of temporal sequences well, we propose a bidirectional recurrent convolutional network for efficient multi-frame SR. Different from vanilla RNNs, 1) the commonly-used recurrent full connections are replaced with weight-sharing convolutional connections and 2) conditional convolutional connections from previous input layers to the current hidden layer are added for enhancing visual-temporal dependency modelling. With the powerful temporal dependency modelling, our model can super resolve videos with complex motions and achieve state-of-the-art performance. Due to the cheap convolution operations, our model has a low computational complexity and runs orders of magnitude faster than other multi-frame methods.

This paper proposes a bidirectional recurrent convolutional network for multi-frame super resolution. In a sense the paper formulates the problem in a way that I think most people familiar with RNNs would consider as a straightforward and sensible way to formulate the problem. The approach appears novel. The Dong et al. ECCV 14 work on learning deep convolutional networks for image super resolution appears to indeed be the most relevant recent work, but it was applied to static frames. This work seems like a very natural extension in the general trajectory of deep convents for super resolution. This combined with the fact that this work provides an evaluation of reasonable quality makes the paper acceptable for NIPS in my view.The most interesting aspect of this work is perhaps the exploration in Table 2 where the effect of feedforward convolutions, recurrent convolutions and conditional convolutions are evaluated. The statements about MATLAB vs Python when considering different explanations for the differences in run times seems like a side issue compared to the more important question of GPU acceleration. Modern convolutional neural networks are almost always implemented using GPUs in state of the art systems. This paper states on line 362 that the implementation of the approach presented here is in Python, but there is no discussion either way concerning the use of GPU acceleration.Please clarify this issue. If a GPU was not used, this method could be dramatically faster. If a GPU was used, then the comparison with prior work really needs to be cast in that light. Many prior methods are likely amenable to GPU acceleration as well.Accurate motion estimation in particular is given as the traditional bottleneck for non-RNN based methods. Such techniques could potentially be quite effectively accelerated with GPU methods.Language Issues:Abstract:* Considering that recurrent neural network[s] (RNNs) can... * Different from vanilla RNN[s]... * conditional convolutional connections from previous input layers to [the]current hidden layer are added for enhancing visual-temporal dependency mod[el].Please proof read the body text for other language issues.Conclusions: In the future, we will [perform] comparison[s] with [other] multi-frame SR methods


Gradient-free Hamiltonian Monte Carlo with Efficient Kernel Exponential Families
We propose Kernel Hamiltonian Monte Carlo (KMC), a gradient-free adaptive MCMC algorithm based on Hamiltonian Monte Carlo (HMC). On target densities where classical HMC is not an option due to intractable gradients, KMC adaptively learns the target's gradient structure by fitting an exponential family model in a Reproducing Kernel Hilbert Space. Computational costs are reduced by two novel efficient approximations to this gradient. While being asymptotically exact, KMC mimics HMC in terms of sampling efficiency, and offers substantial mixing improvements over state-of-the-art gradient free samplers. We support our claims with experimental studies on both toy and real-world applications, including Approximate Bayesian Computation and exact-approximate MCMC.

Summary: The paper is primarily concerned with a Bayesian inference setting in which only unbiased estimates of an unnormalized density are available. The authors propose using kernel methods to approximate the unnormalized density as well as the the gradient of the log-probability in order to perform approximate HMC. The paper uses approximate versions of score matching to approximate the log-probability. Because standard score matching scales poorly with the number of samples being used (that is, the number of samples in the Markov chain thus far), the authors propose only using a random subset of the the samples (KMC lite) or a finite approximation of the infinite-dimensional RKHS (KMC finite). The paper has a number of intriguing ideas and potentially very useful methodologies for pseudo-marginal and ABC-type settings. However, there a substantial issues in the presentation of the methods and in the experimental validations. Major issues: 1. A crucial idea in the paper is to use (approximate) score matching to construct a nonparametric estimate to the (gradient of the) log-probability. However, the authors only describe score matching in the appendix, but at the same time repeatedly refer to the key score matching equation in that appendix. This is rather poor form, and room for the content of appendix A.1 needs to be found in the main paper. More generally, the authors have a bad habit of leaning heavily on the reader to understand a number of other recent papers, in particular their refs [12, 13, 14], and to a lesser extent [3]. I'm sympathetic to the fact that the NIPS paper format offers very limited space, but some of the relevant material (e.g., the details of the experimental setup of [12]), could easily to put in the supplementary material. Extra room could be found by, e.g., removing the proof sketch of Prop 3 and the useless paper outline at the end of the intro. 2. The authors state (line 94+): "we generally assume that [the density of interest] \pi is intractable, i.e. that we can neither evaluate \pi(x) nor \grad \pi(x) for any x, but can compute unbiased estimates of \pi(x)." However, when presenting their KMC method, eq. (3) makes use of the true Hamiltonian, i.e., it requires access to \pi(x). In the experiments section they state they are in fact using PM-MCMC in place of (3), which is fine given the working assumptions. However, this is never made clear earlier, and, moreover, the authors never bother to state the PM-MCMC acceptance criterion they are using. 3. The experiments were difficult to follow and, insofar as I could follow them, were unconvincing. a. Stability in high dims: it was unclear how many samples were used (i.e., the value of t), and how these samples were generated since apparently there was "no MCMC yet." Targeting a standard Gaussian here is extremely uninteresting and not even close to representative of what a real high-dimensional target might look like. Placing a distribution over the eigenvalues of the Gaussian covariance, then generating a number of Gaussians from that distribution would be much more reasonable. Also, it's odd to start from all the origin since in high dimensional centered Gaussians the origin is in a region of increasingly low probability. b. Mixing on a synthetic example: why only investigate mixing? I would expect burn-in to be more problematic since early samples used to estimate pi(x) are going to be unrepresentative. Thus, understanding how KMC performs during burn-in is crucial, and the authors themselves emphasize that the lite and finite versions will perform very differently while in the tails. Small point: min/median/max ESS would be more informative than mean ESS. c. GP classification: This task seems ok, though it would be good to include further experimental details (e.g., model used, source of benchmark samples) in the supplemental material. How much additional computation is required for the selection of kernel width via x-validation? Needing to run this x-validation seems potentially problematic. Why was 45% acceptance targeted? Why does the KMC MMD stop decreasing after ~3000 iterations in Fig 4(left)? Was KMC finite also tested for this problem? d. ABC: I found it extremely difficult to follow the set-up here. What is the "KMC-ABC" algorithm being tested? (As in 3c, Alg 1 as written is not applicable.) I eventually figured out that data was sampled from p(y | alpha = theta = 10 * \vec 1), then theta was inferred using the two ABC algorithms. But this was not clear on first reading and further details are missing. E.g., how many data points were used to form the posterior? Even more importantly, the skew-normal distribution used to test KMC-ABC is very simple and unrepresentative of the problems ABC is applied to.Minor points: 1. The fact that KMC lite falls back to a random walk seems like a really nice feature of that approach and the sort of important detail that might convince a practitioner to try out KMC ("when the method doesn't know what to do, it falls back to method I understand will do something sensible!"). Therefore, it would be good to add a sentence at line 301 explaining how/why this reversion to RW MH occurs. 2. It is of course well-known that choosing reasonable kernel hyperparameters is very important, since they typically encode scale/smoothness assumptions. Particularly in the very complicated models for which only unbiased estimate of pi are available or for which only forward simulations can be performed, the smoothness may vary substantially throughout the parameter space. The paper only addresses these sorts of issues in passing at the end of section 5. Other comments: Kamiltonian Monte Carlo is a terrible name. Why not just "kernel HMC"?l.126: missing comma in Hamiltonian l.224: with => are given byRESPONSE TO AUTHORS: I appreciate the authors' thoughtful response and efforts to redo the Gaussian experiments. However, there are substantial enough issues with the presentation of the methods and the experiments that I think the paper should be re-reviewed. I very much like the paper - the ideas in the paper are very interesting and they deserve to be presented in a clear manner so that the community can benefit from them as much as possible.


Spectral Learning of Large Structured HMMs for Comparative Epigenomics
We develop a latent variable model and an efficient spectral algorithm motivated by the recent emergence of very large data sets of chromatin marks from multiple human cell types. A natural model for chromatin data in one cell type is a Hidden Markov Model (HMM); we model the relationship between multiple cell types by connecting their hidden states by a fixed tree of known structure. The main challenge with learning parameters of such models is that iterative methods such as EM are very slow, while naive spectral methods result in time and space complexity exponential in the number of cell types. We exploit properties of the tree structure of the hidden states to provide spectral algorithms that are more computationally efficient for current biological datasets. We provide sample complexity bounds for our algorithm and evaluate it experimentally on biological data from nine human cell types. Finally, we show that beyond our specific model, some of our algorithmic ideas can be applied to other graphical models.

This paper proposes to solve the parameter learning problem in tree structured HMM using spectral method. Taking advantage of the tree structure, the authors reduced complexity from x^D to x^d where D is the number of nodes in the tree and d is the depth. This represents a significant saving in the case of d << D (for example, for balanced trees we have d = O(log(D)).The paper is well written, where both the goal and the approach is well motivated and the model is clearly explained as well. The authors summarized their contribution in two techniques called the skeletensor and the product projection, both of which are new to me.The experimental results seem strong, although I'm not familiar with the settings and the dataset. One thing that is objectionable is that SMF seems too weak as far as a baseline is concerned. Perhaps regular EM makes a good baseline, since it seems to me that EM can also use the tree structure instead of working in an exponentially large state space by expanding the tree. So I'm not convinced that EM should be dismissed as very slow without providing more arguments. I understand that space is constrained, but experiments on synthetic data, with generic evaluations (like parameter distance, loglikelihod) would help people in the ML community not familiar with the biology task better evaluate this method. Overall, while I'm not entirely sure how important tree HMM for the application, this paper made clear, original progress on an interesting problem, and the techniques developed can probably be used in other settings. This paper should be accepted.


A Reduced-Dimension fMRI Shared Response Model
Multi-subject fMRI data is critical for evaluating the generality and validity of findings across subjects, and its effective utilization helps improve analysis sensitivity. We develop a shared response model for aggregating multi-subject fMRI data that accounts for different functional topographies among anatomically aligned datasets. Our model demonstrates improved sensitivity in identifying a shared response for a variety of datasets and anatomical brain regions of interest. Furthermore, by removing the identified shared response, it allows improved detection of group differences. The ability to identify what is shared and what is not shared opens the model to a wide range of multi-subject fMRI studies.

The manuscript "A Reduced-Dimension fMRI Shared Response Model" provides a model for correspondence across multi-subject fMRI datasets, related to the popular hyper-alignment model.I appreciate the fact that the paper does a good effort to build a sound mathematical model behind hyper-alignment related algorithms and provide good validation. Given the popularity and promises of such techniques, this work in important. The manuscript is in general clear and well written, although some details about the validation could be added. The work is original: it does not claim to be a radical departure from the prior art, which often reflects a lack of knowledge of the literature, but explains well how it relates to previous work, and how it improves significantly upon it. Below I give specific comments that I believe that authors can use to improve their paper. To me, an important aspect of the paper is that it reframes the popular hyper alignement technique and improves upon. I think that the formulation of paper does not make it appear enough. The term "hyper alignment", that is well known to the fMRI world, should be used more, in particular in the abstract, introduction, and conclusion. I understand that the authors may fear that it could make their work look incremental, however, nothing is built from a void, and stressing the relationship to previous work is a strength, given that this contribution improves upon it. I believe that experiment 1 does not show what the authors claim that it shows, in particular the sentence 'This supports the claim of a shared response with distinct functional topographies across subjects.'. Indeed, smoothing reduces the degrees of freedom of the noise. Thus is makes the variance of a comparison across subjects decrease. If there is common information across subjects, the correlation increases mechanically as long as the amplitude of the common information decreases less than the amplitude of the noise (as in the matched filter theorem). This can be shown with very simple simulations using random signals. Thus the increase of correlation across subjects is actually quite trivial. The problem is that smoothing and SRM are both transformations that shrink the volume of the accessible space. The smaller this volume, the more likely correlations are to be strong. In this regard, I do not think that experiment 1 is telling us much. I think that the limitation of correlation as a validation metric appears in the fact that the authors fully explore the logic of maximizing correlation. Indeed, it would be interesting to increase smoothing, and decrease k, until a peak in correlation is reached. Both of these cases would lead to exploring a very restricted signal space, and our intuition tells us that we would probably be throwing out the baby with the bath water. I think that the paragraphs on experiment 1 need to be significantly reworked and downplayed, as they do not provide much evidence that SRM is specific to common signal across subjects. One interesting variant of the experiment would however be to show that when varying SRM parameters, the peak of correlation is higher than when varying smoothing. This would show that SRM is a better filter than smoothing to extract common information.In experiment 2, what was the smoothing used by for the spatial correspondence methods (Talairach and MNI)? Is an optimal in classification reached through varying this parameter?In experiment 2 and 3, more details should be given on how PCA and ICA are used. Right now, it is not possible to reproduce the experiments. In particular, I don't understand the difference between the performances of ICA and PCA, given that 1) the learner used (SVM) is rotationally-invariant and 2) ICA and PCA only differ by a rotation.As a side note, in experiment 3, the sub-space spanned by ICA and PCA are the same, thus it makes sense that they perform the same.The authors have shown that the dimensionality reduction of their SRM method what a strong benefit to prediction accuracy. I am curious to know what dimensionality they used for the ICA and PCA approaches, and if the results of the ICA and the PCA could be improved by varying this dimensionality.


Fast, Provable Algorithms for Isotonic Regression in all p -norms *
Given a directed acyclic graph G, and a set of values y on the vertices, the Isotonic Regression of y is a vector x that respects the partial order described by G, and minimizes x − y , for a specified norm. This paper gives improved algorithms for computing the Isotonic Regression for all weighted p -norms with rigorous performance guarantees. Our algorithms are quite practical, and variants of them can be implemented to run fast in practice.

The problemThe paper studies the Isotonic Regression in $\ell_p$-norms, $1\leq p \leq \infty$. Given a DAG $G(V,E)$ and observations $y\in \bR^{|V|}$, and a weight vector $w$, the isotonic regression is the following minimization problem (which is shown in line 053) \begin{eqnarray} \min_x \|x-y\|_{w,p} \mbox{ such that } x_u \leq x_v \mbox{ for all } (u,v)\in E, \end{eqnarray} where $\|\cdot\|_{w,p}$ is the weighed $\ell_p$-norm. The resultsAllowing a small error $\delta$ from the optimal result, a bound of $O(m^{1.5}\log^2 n \log(npw_{max}^p/\delta))$ on the time complexity which holds with high probability for $1\leq p < \infty$ is shown in Theorem~2.1. For $p<\infty$, the time complexities of this paper are compared with that from previous works which require exact solutions in Table 1. For $p=\infty$ and a variant called the Strict Isotonic Regression (which is defined in line 079), upper bounds of the time complexity to compute exact results are shown in Theorems 1.2 and 1.3, respectively. The time complexity bounds shown in the aforementioned theorems improve the previous results, except for an $\ell_1$ bound in two dimensional space ($V \subset \bR^2$). For $\ell_1$-norm, there is an additional constraint on the number of edges $|E|$.The authors transform the original regression problem to an instance which can be solved by an approximate interior point algorithm called \textsc{ApproxIPM}. By showing the efficiency and the accuracy of a critical subroutine of \textsc{ApproxIPM} called \textsc{BlockSolve} which is designed to compute an approximate Hessian inverse, the proposed algorithm achieves a better time complexity for $1\leq p<\infty$. The contribution of this paper is that the authors generalize a result for linear programs in [23] to $\ell_p$ objectives, and they also provide an improved analysis. For the $\ell_\infty$ Isotonic Regression and the Strict Isotonic Regression, the authors reduce the previous problems to Lipschitz Learning problems defined in [29] and apply the algorithms in [29] to compute the solutions.The paper also provides preliminary experiments on the proposed algorithm, which are listed in Table 2. CommentsThe theoretical part of this paper is an incremental work. The main contributions of this work are the reductions of the problems and the design and the analysis of the critical subroutine \textsc{BlockSolve} which is used to compute an approximate Hessian inverse efficiently. Most of the mathematical techniques used in the analysis can be found in convex optimization, interior point method, and the referenced papers mentioned in this work.It is hard to classify this paper as a theoretical work or an experimental work. The experiments shown in Table 2 are preliminary, and there is no comparisons with other state-of-the-art algorithms. On the other hand, the main algorithm and its analysis ideas are not mentioned clearly in the main body of the paper, although they can be found in the supplementary file. The paper might need restructuring for better presentation. Typos and undefined notations- In line 056, it should be $m \geq n-1$ for a connected graph, rather than $n \geq m-1$. - In line 342, the failing probability should be $n^{-3}$, rather than $n^3$. - In line 663, $\textsc{Solve}_{H_F}$ is not defined, thus making the proof of Theorem 2.7 hard to follow. - In line 716, \textsc{Solve} is not defined, thus making the proof of Lemma A.5 hard to follow. - In line 722, there might be an unnecessary z. QualityFor the theory part of this work it is an acceptable paper. The experimental results are not good enough for publish. The entire presentation of the paper (considering only the main body without the supplementary file) falls between a border line paper and a weakly rejection. ClarityThe algorithm and its critical analysis and analyzing idea are missing in the main body of the paper, making readers not easy to grasp a good understanding to this work.  Originality and SignificanceThis is an incremental work on the Isotonic Regression.


Recursive Training of 2D-3D Convolutional Networks for Neuronal Boundary Detection
Efforts to automate the reconstruction of neural circuits from 3D electron microscopic (EM) brain images are critical for the field of connectomics. An important computation for reconstruction is the detection of neuronal boundaries. Images acquired by serial section EM, a leading 3D EM technique, are highly anisotropic, with inferior quality along the third dimension. For such images, the 2D maxpooling convolutional network has set the standard for performance at boundary detection. Here we achieve a substantial gain in accuracy through three innovations. Following the trend towards deeper networks for object recognition, we use a much deeper network than previously employed for boundary detection. Second, we incorporate 3D as well as 2D filters, to enable computations that use 3D context. Finally, we adopt a recursively trained architecture in which a first network generates a preliminary boundary map that is provided as input along with the original image to a second network that generates a final boundary map. Backpropagation training is accelerated by ZNN, a new implementation of 3D convolutional networks that uses multicore CPU parallelism for speed. Our hybrid 2D-3D architecture could be more generally applicable to other types of anisotropic 3D images, including video, and our recursive framework for any image labeling problem.

The paper proposed to use a composition of a ConvNet with 2D filters and a ConvNet with 3D filters to improve the quality of neural boundary prediction. The authors motivate their architectural choice by the anisotropy (x-y vs z) of the sequential EM scans often used in Neuroscience. Their methods clearly outperforms the previous method (only using 2D filters) on this task which is important for their application.The quality of the presentation could be easily improved and presentation could be streamlined. The method employs previously suggested techniques such as the recursive (composition) of nets and the application of 3D filters for 3D data sets.The paper has unfortunately some problems. The paper refers often to the implementation ZNN (apparently theirs) that is mentioned to be crucial to get it efficient on the size of the network and the amount of data. However, there is no reference provided nor sufficient details are provided to reproduce it. I suggest to actually provide concise details on the essential parts of the implementation and also on whether the software will be released in some way. Otherwise the paper is full of general design choices used to develop ZNN that are not precise enough. The captions to the figures should be more descriptive, as important information is missing to understand them.Details: l80: no reference for ZNN is provided. The reader has no idea what ZNN is. Is it an algorithm or a particular implementation? l177: the description of the Rand index is not sufficiently clear. There is a reference, but the paper should be self-consistent. Please provide the necessary formula. Section 3: This section does not really help to understand the paper. ZNN is not described in enough detail to reproduce the results. I suggest to give more precise details, or provide a reference to the resource where details can be found. Figure 3: color code? l393: In Fig 1 and the rest of the paper you have not used net1 and net2, please be consistent l396: As a comment: recursive seems not to be the right term for because it is not the same network that is recursively applied. It is the composition if vd2d an vd2d3d, although they share the same first layers. Is the vd2d part actually the same or just copied after the initial training and then refined independently?