A Universal Primal-Dual Convex Optimization Framework
We propose a new primal-dual algorithmic framework for a prototypical constrained convex optimization template. The algorithmic instances of our framework are universal since they can automatically adapt to the unknown Hölder continuity degree and constant within the dual formulation. They are also guaranteed to have optimal convergence rates in the objective residual and the feasibility gap for each Hölder smoothness degree. In contrast to existing primal-dual algorithms, our framework avoids the proximity operator of the objective function. We instead leverage computationally cheaper, Fenchel-type operators, which are the main workhorses of the generalized conditional gradient (GCG)-type methods. In contrast to the GCG-type methods, our framework does not require the objective function to be differentiable, and can also process additional general linear inclusion constraints, while guarantees the convergence rate on the primal problem.

The authors presents a framework for the analysis of constrained convex optimization based on Fenchel duality. Rather than the traditionnal proximal operator, this analysis relies on the the sharp operator, a linear minimization oracle, whose computation is generally simpler. The authors formulate a dual problem for which they derive the proximal gradient step as well as a condition on its Hoelder constant. They then introduces two algorithms to solve the dual problem, the second adding a Nesterov-like acceleration scheme. Their methods are said universal as they rely on a line search to automatically adapt to the Holder constant of the dual objective's gradient. The authors present an analysis of the average number of steps involved in this line search. Finally they present several numerical experiments in which they illustrate the advantages of their methods over the Frank-Wolfe algorithm.The paper seems very sound and rely on different state-of-the-art methods. They provide satisfying results on the computation burden of their line-search step.The paper is rather clear despite the complexity of its content. The paper might be difficult to read for those unfamiliar with both universal gradients and generalized conditional gradient methods as the authors cover those notions rather briefly, but the authors do cite the appropriate references. Some graphs are hard to read and the captioning could be improved however.The paper does not propose an entirely novel method as the algorithms combine several existing methods (universal gradients, GCG methods, FISTA acceleration). Those methods however combine rather gracefully into a simple enough algorithmic scheme, and the use of both universal Hoelder inequality and the sharp operaor is far from ubiquitous in the communityThe results are not as encouraging as they could be. The unaccelerated method does not seem to perform better than Frank-Wolfe, even with constant step, despite its significant increase in both theoretical and implementation complexity. The accelerated method seems to have a slight edge on Frank-Wolfe with line-search however. Even if in the second setting their algorithm seems more promising, its bad graph formatting makes it hard for the reader to determine the extent to which the accelerated method outperforms Frank-Wolfe. A plot of log((error_min-error)/error_min) where error_min is the RMSE/NMAE for a large number of iterations would really help to assess the performance.The authors finally presents two settings in which FW-like methods do not apply. As the authors do not compare their results with any baseline it is hard to reach any conclusion, beyond the great acceleration provided by their second scheme. Even naive baselines (like projected sub-gradient) would be useful to assess the quality of the results.It is however rather satisfying to see an empirical confirmation of the authors' prediction regarding the number of call involved in the line search.In the end I think the ideas proposed by the authors are quite good, but the paper fail to deliver a convincing case for their use against conditionnal gradient methods yet. A better vizualization of the performance of their algorithms in the experimental part could however convince me otherwise.l53 dou you mean "may not" rather than cannot? Figure 2: bad formating, hard to read Figure 2 & 3: captioning is lacking, it is not clear to me which is what


Inverse Reinforcement Learning with Locally Consistent Reward Functions
Existing inverse reinforcement learning (IRL) algorithms have assumed each expert's demonstrated trajectory to be produced by only a single reward function. This paper presents a novel generalization of the IRL problem that allows each trajectory to be generated by multiple locally consistent reward functions, hence catering to more realistic and complex experts' behaviors. Solving our generalized IRL problem thus involves not only learning these reward functions but also the stochastic transitions between them at any state (including unvisited states). By representing our IRL problem with a probabilistic graphical model, an expectation-maximization (EM) algorithm can be devised to iteratively learn the different reward functions and the stochastic transitions between them in order to jointly improve the likelihood of the expert's demonstrated trajectories. As a result, the most likely partition of a trajectory into segments that are generated from different locally consistent reward functions selected by EM can be derived. Empirical evaluation on synthetic and real-world datasets shows that our IRL algorithm outperforms the state-of-the-art EM clustering with maximum likelihood IRL, which is, interestingly, a reduced variant of our approach.

The authors propose an IRL method that learns locally consistent reward functions from expert demonstrations. To do so, they suppose the existence of a transition function for switching between reward functions which depends on the rewards and the current state and allows them to build a graphical model of their problem.Their algorithm consists in maximizing the log-likelihood of the expert's demonstrated trajectories depending on some parameters which are the original distributions of states and rewards, the local rewards and the transition function between rewards. To do so, they use the expectation-maximisation (EM) method. Then, via the Viterbi algorithm, they are able to partition the trajectories into segments with local consistent rewards.Strengths of the paper:1. The authors leverage existing and classical methods from the machine learning and optimization fields such as EM, Viterbi, Value iteration and gradient ascent in order to build their algorithm. This will allow the community to easily reproduce their results. 2. The experiments are conducted on synthetic and real-world data. They compare their method to MLIRL which does not use locally consistent rewards and which is the canonical choice to compare to as their algorithm is a generalization of MLIRL. The results presented show the superiority of their method over MLIRL. 3. The idea presented by the authors is original as far as I know.Weaknesses of the paper:1. The paper is very dense ( the figures are incorporated in the text) which makes the reading difficult. 2. The algorithm proposed needs the knowledge of the dynamics and the number of rewards. The authors, as future works, plan to extend their algorithm to unknown number of rewards, however they do not mention to get rid off the knowledge of the dynamics. Could the authors comment on that as some IRL algorithms do not need a perfect knowledge of the dynamics? 3. The method needs to solve iteratively MDPs when learning the reward functions. For each theta in the gradient ascent a MDP needs to be solved. Is this prohibitive for huge MDPs? Is there a way to avoid that step? The action-value function Q is defined via a softmax operator in order to have a derivable policy, does it allow to solve more efficiently the MDP? 4. The authors are using gradient ascent in the EM method, could they comment on the concavity of their criteria? 5. In the experiments (gridworlds), the number of features for the states is very small and thus it is understandable that a reward which is linear on the features will perform badly. Do the authors consider comparing their method to an IRL method where the number of features defining the states is greater? This is the main problem that I have with the experiments, the features used are not expressive enough to consider using a classical IRL method and this can explain why MLIRL performs badly and that its performance does not improve when the number of expert trajectories grows. 6. The performance is measured by the average log-likelihood of the expert's demonstrated trajectories which is the criterion maximized by the algorithm. I think that a more pertinent measure would be the value function of the policy produced by the optimization of the reward obtained by the algorithm. Could the authors comment on that and explain why their performance metric is more appropriate?Summary: This paper presents a new idea consisting in learning locally consistent rewards in order to solve the IRL problem. The method is original and relies on classical machine learning methods. However, I have some concerns on the efficiency of the method for large MDPs and when the dynamics is not provided.


Learning with a Wasserstein Loss
Learning to predict multi-label outputs is challenging, but in many problems there is a natural metric on the outputs that can be used to improve predictions. In this paper we develop a loss function for multi-label learning, based on the Wasserstein distance. The Wasserstein distance provides a natural notion of dissimilarity for probability measures. Although optimizing with respect to the exact Wasserstein distance is costly, recent work has described a regularized approximation that is efficiently computed. We describe an efficient learning algorithm based on this regularization, as well as a novel extension of the Wasserstein distance from probability measures to unnormalized measures. We also describe a statistical learning bound for the loss. The Wasserstein loss can encourage smoothness of the predictions with respect to a chosen metric on the output space. We demonstrate this property on a real-data tag prediction problem, using the Yahoo Flickr Creative Commons dataset, outperforming a baseline that doesn't use the metric.

In this paper, the authors propose to use the wasserstein loss for data fitting in multilabel/multiclass learning. This loss can encode specific losses for between classes error and will promote meaningfull errors (such as error between dog races) instead of costly errors( for instance the recent Google Gorilla/people).The authors discuss the use of a regularized version of wasserstein distance (mainly for computational reasons).They propose 2 potential extensions that can handle the difference of mass between the data and the predicted values. Short numerical experiments show the interest of the approach.It is an good paper that propose a novel data fitting term that can encode error cost between classes. Numerical experiments seem encouraging. The rebuttal of the authors clarified a lot of my questions but there are problems that must be addressed before acceptance.-The use of Wasserstein to encode between class loss is elegant but one can wonder if it's not a bit optimistic. Indeed the loss correspond totheaffectation between class that minimize the loss. Intuitively, one would like to minimize the maximum loss over the residue |h()-min(h,y)| in order to avoid the worst case scenario (gorilla/human discussed above). Note that using the maximum loss on the residue would also boil down to TV for multiclass classification and 0-ground metric. This should be at least discussed in the introduction (with a figure?) since Wasserstein is a complex tool that is difficult to interpret.- strategy 4.2.1 seem to just be lost space. If it has been tested and is inferior to the 4.2.2, then the authors should just refer to it in one or two sentences and spend more time detailing the model and the optimization problem.- Add the demonstration for proposition 4.1 and 4.2 in the supplementary material. A discussion about the convergence of the fixed point iteration is also required since it will not provably converge unless the operation is a contraction mapping.- Thanks to the rebuttal, we now know what the model of h(x) is (linear+softmax). This should appear clearly in the paper along with an equation giving the final optimization problem. This must be followed by a quick discussion on the chosen optimization algorithm and a discussion about the type of problem (is it convex? I doubt it). Not onlyis it necessary for reproducible research but it will clarify the proposal.- While you will doubtfully have time to add it in the final version, numerical experiments would benefice from comparison to other multilabel approaches (binary relevance for instance). One way to illustrate the strength of the proposed approach is to use a performance measure independent to the wasserstein (unlike K-cost that will obviously be better for wasserstein minimisation). For instance flicker you could use a clustering of the tags and you could measure the error using inter-cluster errors. If the clustering encode semantically similar tags together, your proposed approach will clearly perform better that other divergence loss (l1,l2,KL) since it will promote the semantic relations encoded in the metric.


Scale Up Nonlinear Component Analysis with Doubly Stochastic Gradients
Nonlinear component analysis such as kernel Principle Component Analysis (KPCA) and kernel Canonical Correlation Analysis (KCCA) are widely used in machine learning, statistics and data analysis, but they cannot scale up to big datasets. Recent attempts have employed random feature approximations to convert the problem to the primal form for linear computational complexity. However, to obtain high quality solutions, the number of random features should be the same order of magnitude as the number of data points, making such approach not directly applicable to the regime with millions of data points. We propose a simple, computationally efficient, and memory friendly algorithm based on the "doubly stochastic gradients" to scale up a range of kernel nonlinear component analysis, such as kernel PCA, CCA and SVD. Despite the non-convex nature of these problems, our method enjoys theoretical guarantees that it converges at the rateÕ(1/t) to the global optimum, even for the top k eigen subspace. Unlike many alternatives, our algorithm does not require explicit orthogonalization, which is infeasible on big datasets. We demonstrate the effectiveness and scalability of our algorithm on large scale synthetic and real world datasets.

This paper proposes a computationally efficient way of scaling kernel non-linear componentanalysis. Previous attempts of scaling kernel PCA-like eigenproblems in machine learningresorted to Nystrom-variant subsampling or random feature linearization of the problem. Each ofthese methods works effectively in certain settings but the number of samples/features may beprohibitively large in some settings.This paper proposes a doubley-stochastic gradient algorithm for solving kernel eigenvalue problems encompassing kernel PCA, CCA, and SVD. It is called doubly-stochastic because sampling is performed in the data samples and the random features (for this paper, limited tostationary kernels whose Fourier transform is well-defined). The paper claims convergence rate of $\tilde{O}(1/t)$ to the global optimum for the recovered eigensubspace (in terms of principal angle), but with a caveat being that the step sizes are chosen properly and the mini-batch size is sufficiently large.Quality: The paper is reasonably well-written with no glaring grammar errors/typos.Clarity: Clear in most parts, some suggestions for improvement: - In Line 300: "which outside the RKHS" -> "which is outside the RKHS" - Please provide the citation for the "median trick" - just to make the paper self-contained. - Please provide the dimensionality of the datasets used in the experiment. For example, the synthetic dataset is mentioned without such information. - Figure 1 and Figure 2 and Figure 3 are hard to look at, when printed on the paper. - Equation (6), Line 305: the Part II should read $ 2 | \tilde{g}_t(x) - h_t(x) |^2 $Originality: The idea of doubly-stochastic gradient descent has appeared in the convex setting where the objective function can be expressed as a convex function (kernel SVM, etc.). This paper attempts to prove the global convergence rate for the principal angle bound on the recovered kernel eigensubspace.Significance: Kernel eigenvalue problems are notoriously hard to scale and achieving high accuracy is challenging. Addressing this problem is a worthy effort.


Maximum Likelihood Learning With Arbitrary Treewidth via Fast-Mixing Parameter Sets
Inference is typically intractable in high-treewidth undirected graphical models, making maximum likelihood learning a challenge. One way to overcome this is to restrict parameters to a tractable set, most typically the set of tree-structured parameters. This paper explores an alternative notion of a tractable set, namely a set of "fast-mixing parameters" where Markov chain Monte Carlo (MCMC) inference can be guaranteed to quickly converge to the stationary distribution. While it is common in practice to approximate the likelihood gradient using samples obtained from MCMC, such procedures lack theoretical guarantees. This paper proves that for any exponential family with bounded sufficient statistics, (not just graphical models) when parameters are constrained to a fast-mixing set, gradient descent with gradients approximated by sampling will approximate the maximum likelihood solution inside the set with high-probability. When unregularized, to find a solution ϵ-accurate in log-likelihood requires a total amount of effort cubic in 1/ϵ, disregarding logarithmic factors. When ridge-regularized, strong convexity allows a solution ϵ-accurate in parameter distance with effort quadratic in 1/ϵ. Both of these provide of a fully-polynomial time randomized approximation scheme.

Summary: The paper proposes a novel approach to computationally efficient maximum likelihood learning in exponential families. In general, finding the maximum likelihood solution is intractable. From a convex optimization perspective, the sticking point is the need to calculate an integral wrt the currently proposed EF parameter. By assuming that MCMC is fast-mixing for all allowed parameters, the author(s) are able to show that the integrals needed for proximal gradient descent can be calculated with sufficient precision that, when combined with the results of Schmidt et al. (2011), a fully-polynomial randomized approximation scheme for calculating the MLE can be obtained. Both the convex and strongly convex cases are considered, which lead to different types of guarantees: the former on the likelihood error, the latter on the parameter error. A simple experiment demonstrating the theory is also included.Overall, a high quality piece of work. The approach is well-motivated as an alternative to relying on small-treewidth assumptions. The theoretical analyses appear solid, though I haven't checked the proofs in detail. The proofs are essentially a matter of combining the MCMC mixing assumption with concentration inequalities and the results of Schmidt et al. The paper is thus a combination of existing techniques, though the combination is nontrivial. I appreciated the honesty with which the limitations of the approach were discussed. In particular, a thoughtful conclusion analyzed weaknesses of the approach, ways in which the theoretical assumptions are overly pessimistic, and directions for future work. The paper also offers some insight into CD-type algorithms, even though the algorithm that is analyzed is not quite contrastive divergence. The introduction was well-written with substantial references to previous work, but there were a few issues with the clarity of technical presentation. One small but important omission was simply the statement that the log-likelihood is in fact convex: the 2nd and 3rd terms in eq. 1 are clearly convex and the convexity of A(theta) follows from the fact that the hessian of A is the covariance of the sufficient statistics, and thus is PSD. While seemingly minor, the whole paper is premised on the the convexity of f(theta), and so discussing it will help guide the reader through the logic of the paper. A clear summary of the assumptions made in the paper. For example, one assumption never mentioned in the main paper is that the parameter set Theta must be convex. Overall, I found the paper to be an original and substantial contribution. The results to seem to be of limited practical value due to the fact that proving fast mixing of MCMC is extremely challenging. So in a sense the paper has replaced one hard problem (finding the MLE) with another (finding fast-mixing Markov chain). That said, being able to convert one problem into another is often useful, and thus having the results of the paper available to the community is valuable. Other comments: l185: ",w here" => ", where" l192: this paper that => this paper assumes that l207: simple example fast-mixing => simple example of a fast-mixing l281: of of => of l311-12: This sentence is nonsensical


Frank-Wolfe Bayesian Quadrature: Probabilistic Integration with Theoretical Guarantees
There is renewed interest in formulating integration as a statistical inference problem, motivated by obtaining a full distribution over numerical error that can be propagated through subsequent computation. Current methods, such as Bayesian Quadrature, demonstrate impressive empirical performance but lack theoretical analysis. An important challenge is therefore to reconcile these probabilistic integrators with rigorous convergence guarantees. In this paper, we present the first probabilistic integrator that admits such theoretical treatment, called Frank-Wolfe Bayesian Quadrature (FWBQ). Under FWBQ, convergence to the true value of the integral is shown to be up to exponential and posterior contraction rates are proven to be up to super-exponential. In simulations, FWBQ is competitive with state-of-the-art methods and out-performs alternatives based on Frank-Wolfe optimisation. Our approach is applied to successfully quantify numerical error in the solution to a challenging Bayesian model choice problem in cellular biology.

Update after response: with the clarifications mentioned, I'm very happy with the state of this paper as a theoretical contribution to an important problem, and am increasing my score from 7 to 8. ~~~ This paper unifies the Frank-Wolfe based approach for numerical integration with Bayesian Quadrature, developing a numerical integration scheme that provides a full posterior distribution over the result as well as giving good convergence rates. This then also tells us that BQ with optimally-selected points has at least the same convergence rates.It seems that Theorem 1 says only that the BQ weights are no worse than FW weights, and then gives the same convergence rates as [2]. Thus this paper does not establish theoretically that FW(LS)BQ converges faster than FW(LS), but rather that it is no slower. This was not clear to me on first reading and perhaps should be noted more explicitly.The assumption of a finite-dimensional Hilbert space may be standard, but it makes the results somewhat hard to interpret. This assumption seems to come in only through [2, Prop. 1]; [2, Prop. 2] establishes that this same convergence in fact does *not* hold for an infinite-dimensional RKHS. Though section 4.1 notes that any kernel is in practice finite-dimensional, it seems reasonable from the proof of [2, Prop. 1] that as the dimension of the kernel becomes large, the constants C will become quite small and convergence will become slow. It would thus be informative to include (perhaps in the appendix) experiments with a truly finite-dimensional kernel to see if the line search methods perform better in that case. One interesting such experiment might be to use approximations to the exponentiated-quadratic kernel (e.g. random Fourier features) of increasing dimension.On line 279-280: This seems to imply that the main way kernel choice influences the convergence rate is through the constant R. But R = \sup_{x, y \in X} ||\Phi(x) - \Phi(y)||   = \sup_{x, y \in X} \sqrt{k(x, x) + k(y, y) - 2 k(x, y)}. Consider the exponentiated-quadratic kernel of section 4.1. When X is unbounded, R = \sqrt{2} \lambda. One can thus minimize R by minimizing \lambda, but of course that is not the best path to take, and indeed changing \lambda also changes the constants C. In general, some discussion of how C is affected by the kernel choice would be quite interesting.It seems that maintaining the constants ||f||, R, and so on in Theorems 1 and 2 would not be overly cumbersome; though C is of course difficult to determine, it at least satisfies a few simple scaling properties, and so this might provide added understanding in the behavior of the bound under different situations.Lines 345-346 note that "the weights for SBQ are optimal at each iteration, which is not the case for FWBQ and FWLSBQ." Given the design points, don't FW(LS)BQ and SBQ weight those points in the same way?It would be useful to briefly compare the computational costs of the various methods, both asymptotically (which is noted at various points through the paper but not fully compared) and in terms of actual runtimes.Relatedly, it seems that M(n) must grow exponentially in n for the stated FWLSBQ rate to hold, which seems like a substantial computational burden.The conjecture that the convergence rates also apply to SBQ seems reasonable. Given that, is there any practical situation in which FW(LS)BQ would be preferable to SBQ, or should it be used only by conservative practitioners seeking a guaranteed method?


Accelerated Proximal Gradient Methods for Nonconvex Programming
Nonconvex and nonsmooth problems have recently received considerable attention in signal/image processing, statistics and machine learning. However, solving the nonconvex and nonsmooth optimization problems remains a big challenge. Accelerated proximal gradient (APG) is an excellent method for convex programming. However, it is still unknown whether the usual APG can ensure the convergence to a critical point in nonconvex programming. In this paper, we extend APG for general nonconvex and nonsmooth programs by introducing a monitor that satisfies the sufficient descent property. Accordingly, we propose a monotone APG and a nonmonotone APG. The latter waives the requirement on monotonic reduction of the objective function and needs less computation in each iteration. To the best of our knowledge, we are the first to provide APG-type algorithms for general nonconvex and nonsmooth problems ensuring that every accumulation point is a critical point, and the convergence rates remain O 1 k 2 when the problems are convex, in which k is the number of iterations. Numerical results testify to the advantage of our algorithms in speed.

The paper proposes an accelerated gradient-based algorithm for         non-convex composite optimization. Essentially, it provides an         alternative to the existing work of Ghadimi and Lan, 2013. The main         features of the algorithm is to preserve Nesterov's acceleration for         convex problems, while providing convergence to stationary points for         non-convex ones. The paper also presents convergence analysis from         various points of views.          I like the paper and believe that it is a strong contribution.         Nevertheless, there are a few points that could improve the paper         - I found the comparison with Ghadimi and Lan not quite fair. On the one         hand, their approach requires a convex function g and does not apply to          the more general problem considered in the paper. On the other hand,         their convergence results are significantly stronger than simple convergence         to stationary points, since they also provide convergence rates.         - The experimental part is poor compared to the rest of the paper. The purpose         of this experiments should be to convince the reader that acceleration          applies in practice. To settle the case, it would have been good to          simply report the objective function achieved by the different methods as          as function of the number of iterations. Instead, the table reports a         comparison for a specific arbitrary objective function value. Why         this objective function value makes sense is not clear. It is also         not clear that conclusions drawn for this value would generalize to         other ones.          - Finally, I found the experimental setup presented in the         supplementary material (sparse PCA) more interesting than the one         presented in the main paper. sparse PCA is more challenging, and more         interesting. This is of course subjective, but perhaps, the authors         should consider including the sparse PCA experiment in the main paper         instead of the logistic regression one.          Misc comments:         - Please define and comment on the KL assumption in the final version         of the paper, if it is accepted. The paper should be self-content         without having to read the supplementary material.


Less is More: Nyström Computational Regularization
We study Nyström type subsampling approaches to large scale kernel methods, and prove learning bounds in the statistical learning setting, where random sampling and high probability estimates are considered. In particular, we prove that these approaches can achieve optimal learning bounds, provided the subsampling level is suitably chosen. These results suggest a simple incremental variant of Nyström Kernel Regularized Least Squares, where the subsampling level implements a form of computational regularization, in the sense that it controls at the same time regularization and computations. Extensive experimental analysis shows that the considered approach achieves state of the art performances on benchmark large scale datasets.

Summary: The authors study the computational/statistical tradeoff of Nystrom methods in the kernel ridge regression problem in case of random design. They derive optimal finite sample bounds for the excess risk of the estimator (Theorem 1) by choosing the (n,m,\lamda) [n - number of samples, m - number of subsamples in the Nystrom method, \lambda -regularization parameter] appropriately. The paper also gives insight how 'm' plays the role of regularization both from theoretical (computational error term) and practical point of view (numerical experiments). The proof relies on approximation theory and functional analysis; the combination of these results to obtain the bounds is nice and technical. The paper is well organized, the results are clearly presented and relevant.Below I enlist my major comments, improvement suggestions; my minor comments are listed afterwards:-Generally, I would encourage the authors to avoid the 'it is easy to see' type of statements: such _conjectures_ are often error-prone (some additional derivation never hurts).main: -line 280-281: In '1)' is the second term really of the same order as the first and third one? This requirement would imply two (possibly contradictory) equations for 'm'. Please detail and clarify the statement. -line 338-342: The stated update rule via the block matrix inversion lemma is screwed up (in \alpha_t: M_t:(tl)x(tl), C_t:nx(tl), \hat{y}_n: nx1 -- they are not compatible; C_1 is undefined; in D_t, the last B^t could be B_t, ...). Please, provide a precise derivation and formula.supp: -Lemma 3: In the proof of Lemma 3 Prop. 11 is applied, which requires H=H(K) to be a _separable_ Hilbert space. An RKHS is not necessarily separable; thus, assume/guarantee it. A relatively mild sufficient condition is 'separable topological domain with a continuous kernel'; see Lemma 4.33 in [25]. -Lemma 1: --\alpha comes from line 559, thus in line 555 \alpha should be \tilde{\alpha} and \hat{y} should be \hat{y}_n [in accord with Eq. (12)]. From the first term (=pseudo-inverse) an 1/n multiplier appears, from B_{mn} an \sqrt{n} => an \sqrt{n} term seems to be missing from the final expression. ? --line 549: This \hat{y}/\hat{y}_n definition is unclear: \hat{y}_n has already been defined (line 98)... -line 858: Based on the B1 bound, the exponent of C_m in Eq. (15) seems superfluous (i.e., it is equal to 1); the error propagates to line 928 and 963.Minor comments/typos: ==================== main: -||operator||: define/denote that this is the operator norm (not Hilbert-Schmidt norm/...); the same holds for vector norms (it appears in the text as ||v|| and ||v||_{H} as well; I would recommend to use the 2nd form) -line 88: \lambda \ge 0 -> \lambda > 0 (otherwise for example uniqueness might not hold) -line 113: The (K_{mn})_{ji} = K(\tilde{x}_j,x_i) (or K_{mn}=K{nm}^T) definition is missing -line 267: \hat{f}_{\lambda m} -> \hat{f}_{\lambda,m} (comma is missing, see also throughout the paper) -line 280: approah -> approach -line 319: Eq.5 -> Eq.~(5); see also line 374. -line 319: the selected points -> be the selected points -line 320: dimension l -> size l -line 320: please add "t=1,...,T" -line 338: \alpha_t should be \tilde{\alpha}_t -line 344: to compute \tilde{a}_1,...,\tilde{a}_T -> to compute \tilde{\alpha}_1,...,\tilde{\alpha}_T -References: [9,14,18]: the accents are missing from Nystromsupp: -line 515: the this -> this -line 532: ']H' -> _{H} -line 533: Z_m* (the '*' is missing on the left) -line 541: bold K -> K_n (see under Eq. (4)) -line 541: "B_{nm} = \sqrt{nm}S_n Z_m*" -> "B_{nm} = \sqrt{n}S_n Z_m*" ('m' seems to be superflous) -line 541: "G_{mm} = m Z_m Z_m*" -> "G_{mm} = Z_m Z_m*" (m is not needed) -line 542: bold \tilde{K} -> \tilde{K}_n [see Eq. (13)] -line 571: UU* should be U*U (it occurs twice) -line 597: S_n should be S_n*; the same holds in line 604, 861 -line 622: \sqrt{()^2} is superfluous -line 636-638: partition -> subset (twice); see also line 650 -line 644: C_m should be "H -> H (or H_m)" type mapping, not "H -> R^m" -line 646: C_{\lambda} has not been defined (only later) -line 663: the the -> the -line 719: Therfore -> Therefore -line 725: colum -> column -line 855: the accent is missing from 'Nystrom' -line 884: What is r? -line 895-896: The 's' character is already in use (see Assumption 4), please select an other one. -line 935: taking -> take-line 1050: "let denote || ||_HS" -> "let || ||_HS denote" -line 1052: "Let name" -> "Let"; the same holds in line 1063 -line 1087: "Let n, n ... integers" -> "Let n ... integer" -line 1099: ". For any" -> ", then for any" -Please add fullstops (for example) to line 578, 598, 632, 985, 997, and use commas in line 604, 858, ...


Gaussian Process Random Fields
Gaussian processes have been successful in both supervised and unsupervised machine learning tasks, but their computational complexity has constrained practical applications. We introduce a new approximation for large-scale Gaussian processes, the Gaussian Process Random Field (GPRF), in which local GPs are coupled via pairwise potentials. The GPRF likelihood is a simple, tractable, and parallelizeable approximation to the full GP marginal likelihood, enabling latent variable modeling and hyperparameter selection on large datasets. We demonstrate its effectiveness on synthetic spatial data as well as a real-world application to seismic event location.

*Update after the author's rebuttal*The results reported in the rebuttal are very interesting and I very much hope the authors include the comparisons and analyses mentioned there in the final version of the paper (if it gets accepted).=== 0. SummaryThis paper presents an approximation method to Gaussian processes models (mainly focused on the Gaussian process latent variable modelGPLVM), based on the "partitioning" approach, where local GPs are fitted tosubsets of the data. The main claimed contribution is that of an "efficient" marginal likelihood approximation based on linking up the local GPs through a Gaussian MRF.1. Clarity The paper is well written with a clearly stated contribution, that of providing a new approximation for GP inference. 2. Quality, originality and significance The paper seems technically sound as the objective function approximating the marginal likelihood, although unnormalized, has some interesting properties: (a) that of recovering the exact marginal likelihood for a tree-structure over the partitions, and (b) that of approximating the marginals p(y_i) and p(y_i, y_j). There are, however, two major deficiencies of this paper: (i) there is some important missing related work and (ii) it should have compared to a better baseline model.Regarding the first deficiency, scaling up GPs, there have been important advances in the last couple of years or so, most notably the work in [1], and subsequent work by other authors. The work in [2] does provide a probabilistic framework for "partitioning" models, while also learning the data-point assignment within a variational framework. Finally, the work in [3] not only provides a distributed approach for large-scale inference but also addresses GPLVM problems. The authors should not only acknowledge these contributions but also carry out comparisons with at least [3]. This brings up the second deficiency, not comparing to a suitable baseline model. Even if [3] is not used, it seems that the authors compare to vanilla GPLVM. It has been shown that the Bayesian GPLVM of Titsias et al (reference 4 in the paper) is a better model and has the same computational cost as the simpler sparse version of Lawrence (reference 7 in the paper). Therefore, it looks like the authors are using a very weak baseline. It is interesting that the proposed model can be seen as an extension of the BCM. However, it inherits its limitation as a transductive method. Another drawback of the paper is its computational cost, which prohibits its applicability to truly large scale applications (in contrast to the references below [1], [2], and [3]). Indeed, n^2m seems too costly to be considered a practical algorithm for large scale problems, and the size of the datasets of the experiments corroborate that. I do encourage the authors to explore the avenue of scaling further their approach. With regards to the experiments, it will be good to see some results regarding hyper-parameter learning and see how the proposed approach compares to the state of the art. 3. Minor comments - sentence on lines 106-107 does not make sense on its own: "by minimizing the KL divergence to the full GP" - What happened to Q_{22}^{(ij)} when going from the Equation in line 234 to the equations below (including Equation 7)? 4. References [1] James Hensman, Nicolo Fusi, Neil D. Lawrence. in UAI 2013. Gaussian Processes for Big Data. in UAI 2013. [2] T. V. Nguyen and E. V. Bonilla. Fast Allocation of Gaussian Process Experts. in ICML 2014.[3] Yarin Gal, Mark van der Wilk, and Carl Rasmussen. Distributed variational inference in sparse Gaussianprocess regression and latent variable models. In NIPS 2014.


Parallel Correlation Clustering on Big Graphs
Given a similarity graph between items, correlation clustering (CC) groups similar items together and dissimilar ones apart. One of the most popular CC algorithms is KwikCluster: an algorithm that serially clusters neighborhoods of vertices, and obtains a 3-approximation ratio. Unfortunately, in practice KwikCluster requires a large number of clustering rounds, a potential bottleneck for large graphs. We present C4 and ClusterWild!, two algorithms for parallel correlation clustering that run in a polylogarithmic number of rounds, and provably achieve nearly linear speedups. C4 uses concurrency control to enforce serializability of a parallel clustering process, and guarantees a 3-approximation ratio. ClusterWild! is a coordination free algorithm that abandons consistency for the benefit of better scaling; this leads to a provably small loss in the 3 approximation ratio. We demonstrate experimentally that both algorithms outperform the state of the art, both in terms of clustering accuracy and running time. We show that our algorithms can cluster billion-edge graphs in under 5 seconds on 32 cores, while achieving a 15⇥ speedup.

The authors present a parallel version of a classic approximated solution to the Correlation Clustering problem on complete unweighted signed graphs. They prove it preserves the 3 approximation ratio while requiring less peeling rounds and allowing nearly linear speed-up. Relaxing synchronization constraints, they also introduce a variant which improve running time at the cost of introducing additional error compared with the serial solution. These claims are supported by experiments on real world data.In my opinion, this paper addresses a major problem and provide a solution which is simple, but also well founded and efficient as demonstrated both theoretically and practically. Since its inception in 2002, Correlation Clustering has drawn a lot of attention. Yet until recently, most of the efforts have been directed to either improve the approximation ratio or devise heuristics in a serial fashion. This has started to change in the last years [1a,2a,3a] yet the algorithms proposed here compared favourably with the state of the art. Moreover the exposition is clear and the paper well written.On the other hand, a few points could be improved.- First, the main hypothesis (i.e. that the graph is complete) should be more visible. For instance, the title could replace "Big" by "Complete" to be more accurate and descriptive. It should also appear in the abstract, to avoid confusing the reader in a rush. Furthermore, the graphs presented in the experimental sections are clearly not complete. Therefore it should be explained how this was accommodated (for example, considering all missing edges to be negative yield the same practical result than if this was actually the case, but it lower the theoretical approximation ratio from O(log n) to 3 and thus must be carefully justified). Also, the view of ClusterWild! as a procedure on a noisy graph with deleted edges has to be clarified in that perspective of complete graphs (are they transformed into negative edges).- Second the proof of the running time depends crucially of a recent result to upper bound the number of competing threads. This result has not yet been peer-reviewed.- Third, the sketch of the proof of theorem 4 is a bit confusing to me. Namely it is not clear how ignoring edges induces *new* bad triangles in \hat{G} Minor comments: line 117: C_i inter C_j should be equal to the empty set line 124: missing point after clustered line 136: write iff in full line 169, right: clusterID(v) not u line 173: I assumed Gamma(v) is the positive neighbours of v ordered by pi but it's not defined line 221: define speed-up here rather than in line 371 line 248: it's not clear where in the appendix are the full details line 269: should it be intersection? line 273: I assumed p_t is the probability of the event \mathcal{P}_t from line 280: do \mathcal{S}_u,v S and S_u,v refer to the same quantity? line 285: define C_u,v line 296: is "positive" needed to qualify neighbours? It doesn't seem to appear in the following line line 299: shouldn't the outer be P/n line 301: maybe remind that P<\epsilon n/\delta. Is it by the way? Theorem 2 doesn't mention epsilon line 308: is \delta the same as in line 508? line 308: 4 \epsilon / (1 - exp(-4\epsilon)) is equivalent to         1+2\epsilon around 0, giving a factor of (3+4\epsilon) to OPT. Yet the         experiments use \epsilon=0.9, in which case1 + 2 * [4 \epsilon /         (1 - exp(-4\epsilon))]  8.4 instead of 3. line 360: appendix B line 365: it's not clear to me what is the semantic of signed edges in that case Figure 2: the legends are barely readable when printed line 427: is there any way to tell which version to prefer without running both of them? line 431: could local heuristics of [3] be parallelized as well? line 461: [17] was accepted in COLT'15 line 500: typo in round line 693: should it be intersection?[1a] Ahn, K., Cormode, G., Guha, S., McGregor, A., & Wirth, A. (2015). Correlation Clustering in Data Streams. In Proceedings of The 32nd International Conference on Machine Learning (pp. 2237-2246). Retrieved from http://jmlr.org/proceedings/papers/v37/ahn15.html [2a] Bonchi, F., Gionis, A., & Ukkonen, A. (2012). Overlapping correlation clustering. Knowledge and Information Systems, 35(1), 1-32. doi:10.1007/s10115-012-0522-9 [3a] Chierichetti, F., Dalvi, N., & Kumar, R. (2014). Correlation clustering in MapReduce. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD '14 (pp. 641-650). New York, New York, USA. doi:10.1145/2623330.2623743


Inference for determinantal point processes without spectral knowledge
Determinantal point processes (DPPs) are point process models that naturally encode diversity between the points of a given realization, through a positive definite kernel K. DPPs possess desirable properties, such as exact sampling or analyticity of the moments, but learning the parameters of kernel K through likelihood-based inference is not straightforward. First, the kernel that appears in the likelihood is not K, but another kernel L related to K through an often intractable spectral decomposition. This issue is typically bypassed in machine learning by directly parametrizing the kernel L, at the price of some interpretability of the model parameters. We follow this approach here. Second, the likelihood has an intractable normalizing constant, which takes the form of a large determinant in the case of a DPP over a finite set of objects, and the form of a Fredholm determinant in the case of a DPP over a continuous domain. Our main contribution is to derive bounds on the likelihood of a DPP, both for finite and continuous domains. Unlike previous work, our bounds are cheap to evaluate since they do not rely on approximating the spectrum of a large matrix or an operator. Through usual arguments, these bounds thus yield cheap variational inference and moderately expensive exact Markov chain Monte Carlo inference methods for DPPs.

Determinantal point processes (DPPs) have been receiving a significant amount of recent attention in the machine learning literature, due to their ability to model repulsion within realizations of a set of points.This repulsion is specified via a positive definite kernel function.One challenge, however, is learning the appropriate parameters of this kernel, given example point realizations.This estimation is difficult due to a difficult-to-compute normalization constant that, in effect, sums over all possible cardinalities and point configurations.A recent proposal [7] has provided a way to do this by observing that the spectrum can be estimated and used to provide incremental upper and lower bounds, resulting in a provably-correct scheme for "retrospective" Markov chain Monte Carlo (MCMC) of the kernel parameters.The present paper proposes a different type of bound that can also be used for MCMC inference, but that does not require estimation of the eigenvalues.This approach also lends itself to a variational approach to learning, and the paper draws connections to inducing-point methods to scalable inference in Gaussian process models.This is a technically strong and well-written paper.I was not aware of the inequality that provides the lower bound in Proposition 1 and it seems like an excellent fit for this problem.It is a clever way to avoid the estimation issues of [7].I love the connections to GP inducing points, as well as the ability to now perform variational inference.Overall this paper was a pleasure to read.I have two technical concerns that I would like to see addressed, however.First, although I like this approach very much, I do not find it all that compelling that it is a huge computational win to replace power iterations with nonlinear optimization problems of increasing dimension.Second, it is not obvious to me that simple increasing of the cardinality is guaranteed to result in convergence of the bounds, because of the coupled optimization problem.Although I am not certain, I believe this optimization is highly non-convex and so one needs to increase cardinality and also ensure that a global minimum is achieved.This is something that should be addressed directly, because it seems that a local minimum could prevent the procedure from being able to bound away the Bernoulli threshold and make it impossible to take an MCMC step.Presentation Issues: - P7 Figure 1: Please include axis labels for the figures. - The proof of the proposition is the main thing that makes this  paper possible and is the central insight.I would've liked to see  it in the main body of the paper and not the index.- P7 L361: "... the variational lower is ..."


Softstar: Heuristic-Guided Probabilistic Inference
Recent machine learning methods for sequential behavior prediction estimate the motives of behavior rather than the behavior itself. This higher-level abstraction improves generalization in different prediction settings, but computing predictions often becomes intractable in large decision spaces. We propose the Softstar algorithm, a softened heuristic-guided search technique for the maximum entropy inverse optimal control model of sequential behavior. This approach supports probabilistic search with bounded approximation error at a significantly reduced computational cost when compared to sampling based methods. We present the algorithm, analyze approximation guarantees, and compare performance with simulation-based inference on two distinct complex decision tasks.

I think this paper presents a valuable contribution to inverse optimal control and more generally to analysis of decision making and control as probabilistic inference. Incorporating ideas from heuristic search to estimate state marginals under the maxent model is an interesting and thought-provoking idea. The authors also evaluate their method on two interesting real-world datasets, including a football prediction dataset.I think that there is a lot to learn from this work, and I can see some follow-up working making good use of the ideas presented in this paper.That said, I think the paper is seriously hampered by confusing writing and excessive "specialization". For example, it took me several passes through the paper to understand that the authors were actually estimating the model parameters theta, which is mentioned briefly in the experiments section and at the end of Sec 2.2. Most of the paper is concerned with inference, which is indeed the most pressing problem in the maxent IOC framework, but the authors don't really explain this very well in the background section. The main technical sections of the paper are quite heavy on derivation and light on intuition and motivation, which also makes them tough to follow. Overall, I think the paper can be improved dramatically from a more thorough top-down organization that clearly lays out what the authors are trying to do, better explains the maxent model, and clarifies where the proposed component fits in in the context of a practical algorithm.


Bayesian Active Model Selection with an Application to Automated Audiometry
We introduce a novel information-theoretic approach for active model selection and demonstrate its effectiveness in a real-world application. Although our method can work with arbitrary models, we focus on actively learning the appropriate structure for Gaussian process (GP) models with arbitrary observation likelihoods. We then apply this framework to rapid screening for noise-induced hearing loss (NIHL), a widespread and preventible disability, if diagnosed early. We construct a GP model for pure-tone audiometric responses of patients with NIHL. Using this and a previously published model for healthy responses, the proposed method is shown to be capable of diagnosing the presence or absence of NIHL with drastically fewer samples than existing approaches. Further, the method is extremely fast and enables the diagnosis to be performed in real time.

# SummaryThe authors introduce a new method for actively selecting the model that best fits a dataset. Contrary to active learning, where the next learning point is chosen to get a better estimate of the model hyperparameters, this methods selects the next point to better distinguish between a set of models. Similar active model selection techniques exist, but they need to retrain each model for each new data point to evaluate. The strength of the author's method is that is only requires to evaluate the predictive distributions of models, without retraining.They propose to apply this method to detect noise-induced hearing loss. The traditional way of screening for NIHL involves testing a wide range of intensities and frequencies, which is time consuming. The authors show that with their method, the number of tests to be run could be drastically decreased, reducing the cost of large-scale screenings for NIHL.# QualityThe paper is technically sound. Although not detailed, the derivations make sense. However, there is no theoretical justification or discussion, and the empirical evaluation is weak. The authors only compare their active model selection scheme with a "traditional" active learning algorithm, and a random baseline, on simulated data generated from their model (possibly biasing the results). Ideally, it should also be evaluated on real data with true examples of NIHL (not generated from the model), and also compared with other active model selection methods. Moreover, some key issues, such as defining the set of candidate points or candidate models, are not discussed at all.# ClarityThe paper is very well written. The structure is clear, and the relevance and usefulness of the proposed method are well introduced. The application is particularly well explained. A pleasure to read!# OriginalityAs stated by the authors, the problem of active model selection is less well-studied than that of active learning. The proposed method is a variation of exiting ones (maximizing mutual information instead of expected cross entropy, for example), that relies on various existing approximations for its implementation. Its application is also original, and not well studied.# SignificanceThe method gives a significant complexity improvement over existing ones, as it does not require retraining the models for each new candidate. It enables real-time application of the method, which is key to some problems. It also allows to reduce significantly the number of testing points required for NIHL screening, which would have a significant impact on large-scale screening, reducing costs.# Pros* Well written and clear * New method for active model selection, that does not require retraining models * Significant gains (fewer samples required) for an important and practical application# Cons* No theoretical analysis/discussion of the new method * Weak empirical evaluation * No comparison with other active model selection techniques * Evaluation on data simulated from their own model, thus possibly biasing results * No discussion of the problem of generating the set of candidate points or models


Neural Adaptive Sequential Monte Carlo
Sequential Monte Carlo (SMC), or particle filtering, is a popular class of methods for sampling from an intractable target distribution using a sequence of simpler intermediate distributions. Like other importance sampling-based methods, performance is critically dependent on the proposal distribution: a bad proposal can lead to arbitrarily inaccurate estimates of the target distribution. This paper presents a new method for automatically adapting the proposal using an approximation of the Kullback-Leibler divergence between the true posterior and the proposal distribution. The method is very flexible, applicable to any parameterized proposal distribution and it supports online and batch variants. We use the new framework to adapt powerful proposal distributions with rich parameterizations based upon neural networks leading to Neural Adaptive Sequential Monte Carlo (NASMC). Experiments indicate that NASMC significantly improves inference in a non-linear state space model outperforming adaptive proposal methods including the Extended Kalman and Unscented Particle Filters. Experiments also indicate that improved inference translates into improved parameter learning when NASMC is used as a subroutine of Particle Marginal Metropolis Hastings. Finally we show that NASMC is able to train a latent variable recurrent neural network (LV-RNN) achieving results that compete with the state-of-the-art for polymorphic music modelling. NASMC can be seen as bridging the gap between adaptive SMC methods and the recent work in scalable, black-box variational inference.

The paper proposes to learn proposal distributions for sequential Monte Carlo (SMC) in nonlinear state-space models (SSMs). The method (NASMC) parameterizes the proposals as (recurrent) neural networks (RNNs) which are trained to fit the filtering distribution obtained with SMC. The approach is evaluated in three different setups: (a) state inference; (b)Bayesian learning (inference) of model parameters; (c) ML parameter learning with approximate (SMC) inference as inner loop. Results suggest that the the adaptive proposals outperform naive proposals (the prior) and also techniques such as the extended Kalman Particle filter and the Unscented Particle Filter (where the latter two are applicable). For ML learning of NN SSMs the approach performs similar to recently proposed stochastic variational approaches.The paper is well motivated and clearly written. In addressing the question of efficient, scalable inference in latent variable SSMs the paper considers an important and timely question. The proposed solution is not extremely original given that there has recently been a large body of work on "learning inference". Even though much of this work has focused on variational approximations the specific idea of learning a NN proposal for importance sampling -- in the non-sequential case --has been explored e.g. in ref [26], and I think the approach is a particular instance of the cross entropy method but with NNs (e.g. Rubinstein & Kroese, 2004; [9]). Nevertheless, to my knowledge the particular setup in the paper has not been explored before, and more generally, I am not aware of much of the recent "learning inference" work being applied to time series models. Also, the proposed approach seems conceptually simple and general, and therefore of practical value.As far as I can judge the authors have put a good effort into evaluating the proposed scheme in the context of different inference problems, covering different use-cases of SMC (including relatively low-dimensional benchmark systems as well as training a stochastic RNN on music data). On the low-dimensional benchmark data the results suggest that the learned proposal distributions can indeed be advantageous, at least compared to simple proposals such as the prior. The results for learning stochastic RNNs are less clear cut, with NASMC performing roughly on par with models of the same class trained using a variational approximation to the posterior. Two questions the evaluation leaves at least partly open is whether we can actually expect the method to scale to large state spaces and whether it provides any advantages over variational inference networks in practice. I think the experimental section could be strengthened in two ways: Firstly, by investigating how, for a known models, the quality of the learned proposal scales with the complexity of the posterior, in particular with the dimensionality of the latent space. Secondly, it I think it would be useful to include a smaller learning experiment, on which you test different proposals, and also variational inference. In the stochastic RNN learning experiments in 5.4 it seems that neither approach to train models with stochastic latent variables does especially well (compared to RNN-NADE). But the insights that we can gain from this are in my eyes somewhat limited as they confound potential issues due to the choice of model class with potential failures of inference. (A simpler learning task may also reveal the bias of VB in learning that you discuss in section 6.) In general, I think it would be useful to study and discuss limitations and failure cases of the approach in more detail. For instance, one worry that one might have about the approach is the fact that, at least as explored in the paper, the training of the proposal relies on the samples from the posterior which is expensive to sample from exactly. Intuitively, this seems to lead to a chicken-and-egg problem: if the posterior is poorly sampled from in the first place, then a poor proposal will be learned - leading to even worse exploration of the posterior. To reduce variance (and allow online learning) the authors propose to use a samples from a filtering approximation to the true posterior as the target for the learned proposal. Again, it seems to me that this could have undesired and, importantly, difficult-to-detect consequences. Maybe these concerns are unfounded but I think it would be worth addressing them directly.  Some additional comments:* line 192: in what sense is the approach minimizing a global cost?* line 219: h_t is only defined in the appendix* Experimental evaluation: when you have a multimodal posterior does it make sense to look at the difference between the posterior *mean* and the ground truth? (This question applies to both the MD parameterization as well as the unimodal proposal...)* MD networks in section 5.1: did you check whether these networks actually learned multimodal proposals or are the mixtures used e.g. to represent heavy-tailed proposals (which could be parameterized more directly)? Furthermore, you argue that the true posterior is multimodal. It would be very instructive to see whether this multimodality is actually properly explored by NASMC.* how exactly did you use prior knowledge about the dynamics in the parameterization of you proposal networks?* section 5.1 the prior as proposal seems to do really poorly but using the prior dynamics in the context of a learned NN appears to give a big boost - how does that fit together? Maybe having a proposal that corresponds to the prior mean plus some heavy-tailed noise would already give you most of the gain? (I think it would be useful to include such a simple baseline.) Also, looking at the table in the appendix the evidence in favor of RNNs over NNs seems not so clear-cut. The main boost seems to come from incorporating prior dynamics, for either network type. But the difference between RNN-MD(-f) and NN-MD(-f) is actually quite small? [I'm finding the table in the main text almost a bit misleading.] Can you provide your "ground truth" LML estimate? * section 5.2: Why is \delta \theta inferred so poorly when the prior is used as proposal (compared to x)? Why is \theta not part of the latent state z_t (see appendix)? Maybe say why you can't apply EKPF / UPF? (Is this because they rely on gradients?) What does inference for sigma_v look like? I'd suggest including other proposal architectures in this experiment as well.* especially in Fig. 3, but also more generally, I think that a fair comparison of NASMC and a naive proposal such as the prior needs to take into account the computational cost of learning the proposal. Using 10 particles from the prior may perform poorly, but when taking into account the cost of learning the proposal you may be able to afford relatively more samples from the prior...* section 5.4 I assume that the generative model structure is exactly the same as for the model trained with SGVB? Otherwise, it would be hard to compare the results.* It would be very useful to know how sensitive the approach is to the right choice of hyperparameters (learning rates, # of particles for training the proposal, relative training of proposal vs. model network, etc.). Especially in high dimensions is there a danger of the proposal overfitting early in learning?* I think that one nice feature of SMC inference compared to SGVB (the way it is is typically implemented, i.e. with stochastic backprop) is that it can be used with discrete stochastic latent variables. Maybe it's worth pointing this out? * line 412ff, the comment regarding entropy: you can typically use an MC approximation when the entropy is not analytically tractable* Another reference you may want to include: Stuhlmueller et al (2013): Learning Stochastic Inverses


Is Approval Voting Optimal Given Approval Votes?
Some crowdsourcing platforms ask workers to express their opinions by approving a set of k good alternatives. It seems that the only reasonable way to aggregate these k-approval votes is the approval voting rule, which simply counts the number of times each alternative was approved. We challenge this assertion by proposing a probabilistic framework of noisy voting, and asking whether approval voting yields an alternative that is most likely to be the best alternative, given k-approval votes. While the answer is generally positive, our theoretical and empirical results call attention to situations where approval voting is suboptimal.

This paper studies the optimality of "k-approval" voting aggregation using the approval voting rule (i.e., picking the item(s) with highest votes). In particular it studies optimality of this principle when the data is generated under the Mallows model (for different distance functions). The paper's main contributions are theoretical results which indicate that for most distance functions not all alternatives selected under approval voting are MLE optimal. However under the Kendall-Tau distance function (easily the most commonly used distance function with the Mallows model) they are equivalent.Overall I found the paper to be an interesting read with intriguing results. That said, I'm uncertain if it reaches the high bar of acceptance for a NIPS submission. My main concerns are as follows:- (Significance) I am unclear of the impact and significance of this work. What should a practitioner take away from this? What about other researchers working on this topic? This remains unclear to me despite reading the paper closely. As the authors themselves state, approval voting isn't much worse than the MLE alternative in the situations where do they do not match. Furthermore the MLE estimator itself is infeasible to compute for large problems.The secondary set of results (as presented in Sec. 4) aren't very impactful in my opinion given that they discuss (and conjecture) about optimality in extreme regions of the space, which most real problems do not fall under.I would also have liked to see some discussion on why k-approval voting is preferable to approval voting as an feedback elicitation mechanism, given that the latter seems like the cognitively easier and more reliable of the two.- (Originality) I would have also wanted to see some the connection made to the numerous results of approximation quality of rank aggregation under the Mallows model (such as the works by Nir Ailon, Cynthia Dwork and other researchers in the field). Given the decades of work on this problem, there is a rich body of literature to draw from, which can actually explain some of these results.In general while this paper indicates non-optimality of some of these models, it does not indicate how suboptimal the approval voting solutions really are (which would have been a far more intriguing result).- (Quality) In general the empirical results indicate only small differences in performance of the two methods. Given this I would have expected some kind of significance testing or error bar reporting as is standard practice.- (Clarity) The paper was mostly clear. However there were notations and terminologies that were overloaded to the point of confusion. For instance the use of the d_{KT} notation on line 205 which is then followed by a completely different use of the same notation in the next two lines. I was also slightly confused at a few points in the paper with the use of the approval voting ruleterminology (which was designed for unconstrained approval feedback) was used under the k-approval voting problem.In general much of the core ideas and insights are lost behind the sea of notation and consequently aren't made clear to the reader. Saying that non-optimality is due to some information being excluded is fine, but I would like to further understand what this information is and what are some preliminary ideas on feasible alternates that can capture this information.A more minor comment: I also didn't understand the noise characterization for the datasets at the start of section 5. Just because differences are smaller does not make the underlying votes more noisy. Unless there were empirical results that indicated the same I wouldn't make that claim.


On the consistency theory of high dimensional variable screening
Variable screening is a fast dimension reduction technique for assisting high dimensional feature selection. As a preselection method, it selects a moderate size subset of candidate variables for further refining via feature selection to produce the final model. The performance of variable screening depends on both computational efficiency and the ability to dramatically reduce the number of variables without discarding the important ones. When the data dimension p is substantially larger than the sample size n, variable screening becomes crucial as 1) Faster feature selection algorithms are needed; 2) Conditions guaranteeing selection consistency might fail to hold. This article studies a class of linear screening methods and establishes consistency theory for this special class. In particular, we prove the restricted diagonally dominant (RDD) condition is a necessary and sufficient condition for strong screening consistency. As concrete examples, we show two screening methods SIS and HOLP are both strong screening consistent (subject to additional constraints) with large probability if n > O((ρs+σ/τ ) 2 log p) under random designs. In addition, we relate the RDD condition to the irrepresentable condition, and highlight limitations of SIS.

The authors propose a two-stage feature selection approach for linear regression. Candidate features are selected in the first stage and the selection is further refined in the second stage. They define "Strong screening consistency" as a criterion for performing the initial stage, and then investigate the implications of that definition when the screening values are computed by a linear operator.From the content and the references it appears that the authors address the statistics community. My expertise is more in machine learning/computer science, and my review is written from that perspective.My main point is the following:The vector beta to be estimated has 0 value for its ith coordinateiff the ith feature should not be part of the optimal selection. Therefore, identifying the 0 coordinates amounts to finding the optimal selection. My understanding of Definition 2.1 is thatit implies the existence of a threshold on the absolute values of beta_hat coordinates. All and only the zeros of beta are below that threshold. But this means that by trying all p-1 thresholds one can find the optimal solution. Since this problem is known to be NP-hard it is very unlikelythat such strong screening can be computed efficiently. Additional comments:The abstract uses many terms that are defined only much later in the paper. These include consistency, rho, sigma, thao, etc.The idea of (not necessarily consistent) variable screening appears to be, in principle, the same as filters (e.g., Guyon03). It may be a good idea to mention this and give references, or explain the differences.I am not sure that the O(n^2p) complexity (line 60) is acceptable for variable screening. Naive implementations of forward selection take O(n^2p), and fast implementations take O(np). In this sense O(n^2p) is very slow.--------------------------------------------------------------------------------------------Thanks for the rebuttal. It clarified most of the issues I had. I still find strong consistency to be too strong, as it correctly identifies the features to be selected. As explained in the rebuttal, this is an easy case, and may not help the challenge posed by feature selection. Still, I can now appreciate the result much better.Here is a reference to O(np) forward selection algorithm: Maung et,al. TKDE 2015.


Calibrated Structured Prediction
In user-facing applications, displaying calibrated confidence measuresprobabilities that correspond to true frequency-can be as important as obtaining high accuracy. We are interested in calibration for structured prediction problems such as speech recognition, optical character recognition, and medical diagnosis. Structured prediction presents new challenges for calibration: the output space is large, and users may issue many types of probability queries (e.g., marginals) on the structured output. We extend the notion of calibration so as to handle various subtleties pertaining to the structured setting, and then provide a simple recalibration method that trains a binary classifier to predict probabilities of interest. We explore a range of features appropriate for structured recalibration, and demonstrate their efficacy on three real-world datasets.

This paper proposes a method to calibrate structured predictions by minimizing the l_2 loss between true outcome and predictive probability.The approach "re-maps" an existing structured prediction model's predictive distribution for each example to be more calibrated.I am somewhat skeptical and confused by the motives of this work.Calibration seems like a sensible concern for weather forecasts because the user-facing "output" of the forecast is the probability prediction itself.Though the predictions from speech recognition, OCR, scene labeling, etc. support "user-facing" applications, the "output" is not the probability estimate for prediction, but the MAP estimate: i.e., words, labeled scenes, detected faces, etc. Should a user feel more satisfied that their structured predictor is more internally consistent (in terms of calibration) when the end-use application performance (e.g., accuracy) is worse as a result?Can a compelling structured prediction application where calibration concerns seem justified be provided?"More often, the probabilities obtained ... will not be useful in practice, mainly because they will not be calibrated" (line 081)It is absurd to suggest that all of the non-calibrated structured prediction methods currently used in practice are not useful in practice.Figure 1b: the example probabilities do not appear consistent P("land") + P("lano") < P("l***").Much of the description of background work is light on references (in fact, there are zero references in the "Background" section!) and the connections to previous work not fully illuminated.For example, the l_2 loss is also known as the Brier loss and has been fairly extensively studied.The components of decomposition in (3) were originally called uncertainty, tresolution, and reliability [13].It is unclear where e.g., the entropy term comes from -- it does not seem to match with, e.g., the Shannon entropy.The experiments show that the calibration method proposed does indeed provide better calibration than the original structured predictions.The small comparison between decision tree-based recalibration and k-NN-based recalibration seems insufficient.If this idea of nearest neighbor re-calibration is novel, why not provide comparisons also in the classification setting with [15, 18, 14]?Significantly more demonstration and/or discussion of why these previous methods cannot also be applied in some manner to these experiments is warranted.-------------The author feedback argues that quantifying prediction uncertainty is important, which I agree with, but this still doesn't explain why _calibrated_ uncertainty is essential.The authors seem to equate the two, but there are many models that estimate uncertainty without being well-calibrated.The reference provided to a structured prediction application in bioinformatics, "Taxonomic metagenome sequence assignment with structured output models," uses SVM-based methods presumably to optimize accuracy rather than calibrated uncertainty. I'm not sure how this reference is supposed to advance the argument.


Empirical Localization of Homogeneous Divergences on Discrete Sample Spaces
In this paper, we propose a novel parameter estimator for probabilistic models on discrete space. The proposed estimator is derived from minimization of homogeneous divergence and can be constructed without calculation of the normalization constant, which is frequently infeasible for models in the discrete space. We investigate statistical properties of the proposed estimator such as consistency and asymptotic normality, and reveal a relationship with the information geometry. Some experiments show that the proposed estimator attains comparable performance to the maximum likelihood estimator with drastically lower computational cost.

This paper proposes a new method to derive an estimator for discrete value parameters in probabilistic models, which does not require the calculation of the normalizing constant. This avoids a computational bottle neck, which is often of exponential order and makes many problems infeasible. This new estimator minimizes a risk function defined as the localized pseudo-spherical (PS) divergence between the empirical distribution of the dataset and the unnormalized probabilistic model. This risk function can be made to be convex with properly chosen parameters. The paper shows that its proposed estimator is asymptotically efficient and additionally gives an intuitive explanation of this by showing the localized PS divergence can be interpreted as a mixture of alpha-divergences. The method is applied to fully visible and hidden variable Boltzmann machines and the estimator is compared with the MLE.The quality of this paper is very good. It is well written, organized, and technically strong. I think there is a typo in Remark 9 on line 337, which should be just alpha > 0 in the parentheses so that sgn(alpha) doesn't matter.Additionally it would be nice if the simulations compared (both of RMSE and computation time) the proposed estimator to estimators from other methods [5], [6], and [10] instead of just the MLE. There are several issues that should be dealt with.In line 352, the experiment on fully observed Boltzmann machine is done with data dimension d=10. This is not very practical. Note that the claim of this paper is the efficiency, so it should tested on a larger dimensional example, e.g. d greater than 100.Furthermore, depending on the modeling of Boltzmann machine in line 82 and line 88, it is not fully convincing that this method is ready to be applied in deep neural network with large dimension of hidden layer. Furthermore, the author's result only applies to exponentially families, and may not necessarily hold in a mixture family like the Boltzmann machine with hidden layers.The critical assumption for the model to work is that the empirical distribution is restricted on a finite subregion $\cZ$, while the model is defined on domain $\set{0,1}^{n} \supset \cZ$.The authors should make this clear in the statement of their theory. It is noted that in the last line of page 3, the author introduces a trick to compute the weighted normalization factor as\sum_{x}\ind{x\in \cZ}\frac{n_{x}}{n}q(x). It should be mentioned that in generalBoltzmann machine is a mixture model that does not satisfy the constraint that there exists a domain Z \subset \set{0,1}^{n}, such that \frac{n_{x}}{n} = 0, x\not\in Z.   There are some other confusing typos:1. In line 225, r_{\alpha, \theta_{0}}(x) = \bar{q}_{\theta_{0}} (x) should be r_{0}, \theta_{0}}(x) = \bar{q}_{\theta_{0}} (x) as confusing typo. Same typo is in supplementary material. 2. In line 225, "By expanding an equilibrium condition" is not clearly explained in either main text and supplementary material. Could it just be the Taylor expansion?This should be clarified.3. Supplementary material line 92, a typo, \langle \phi \epsilon\rangle should be \langle \psi'_{\theta_{0}} \epsilon\rangle.


Rethinking LDA: Moment Matching for Discrete ICA
We consider moment matching techniques for estimation in latent Dirichlet allocation (LDA). By drawing explicit links between LDA and discrete versions of independent component analysis (ICA), we first derive a new set of cumulantbased tensors, with an improved sample complexity. Moreover, we reuse standard ICA techniques such as joint diagonalization of tensors to improve over existing methods based on the tensor power method. In an extensive set of experiments on both synthetic and real datasets, we show that our new combination of tensors and orthogonal joint diagonalization techniques outperforms existing moment matching methods.

The paper builds on the moment matching methods for standard LDA, providing related algorithms that estimate topic token distributions in a fully specified generative LDA model (one with a specific prior distribution for document length). I found the paper quite clear, the approach interesting, and the results convincing. My only suggestion is to clean up the notation a bit - the zoo of models, variables, and indices makes it more difficult to parse than it should be.Specifically:I don't think the Discrete PCA section is necessary.It seems like the GP model is just a stepping-stone to the more general DICA model for which results are presented, linking DICA to the natural choice of Poisson distribution for L.However, it might be more clear to skip directly to the DICA model, and bring up earlier the meaning of alpha's (topic intensities).At the end of section 4, the authors mention that the violation of the non-negativity constraint is practically significant.What consequence does that have on convergence, stability?This issue seems important enough to expand a bit.Regarding the last point made in Conclusions, is it at least possible to estimate the relative scales of topic intensities given assumptions on D?Minor things:line 137, 669 "low of total" to "law of total"line 298 "some up" to "sum up"line 366 we set b in (4)(?)


The Pareto Regret Frontier for Bandits
Given a multi-armed bandit problem it may be desirable to achieve a smallerthan-usual worst-case regret for some special actions. I show that the price for such unbalanced worst-case regret guarantees is rather high. Specifically, if an algorithm enjoys a worst-case regret of B with respect to some action, then there must exist another action for which the worst-case regret is at least Ω(nK/B), where n is the horizon and K the number of actions. I also give upper bounds in both the stochastic and adversarial settings showing that this result cannot be improved. For the stochastic case the pareto regret frontier is characterised exactly up to constant factors.

This paper addresses the minimax bandit problem in the stochastic setting. The case that is targeted is to bound specifically the regret for a particular arm and characterised how this deteriorates the regret on the remaining arms. The authors provide new theoretical material to prove an upper and lower bound on the repartition of the per-arm regret that match up to constants. The upper bound is based on the proposednew algorithm Unbalanced-Moss which is based on the Moss algorithm. They show that unlike in the expert case, guaranteeing a small regret for a particular arm comes to a greater cost (than in the expert case) for the remaining arms. Finally they design a new version of UCB (unbalanced UCB), that up to a logarithmic factor is 'unfair minimax' optimal and which at the same time and contrary to the MOSS algorithm possess asymptotic optimal problem-dependent regret. The paper addresses a problem of importance and seems in the most part of high value. My main concern is that I am not sure about the correctness of the lower bound proof. The question raised there should definitely be addressed.Some part of the proofs are quite compact and makes them hard to verify: passage (c)&(d) of proof of theorem 3. More intuition should be given to ease the reading of the proofs.--About the lower bound:Equation in line 143: Shouldn't it be an inequality instead of the last equality, as \epsilon_k is defined with respect to R_k which is a sup on R_\mu?  In line 126, it is assumed that \mu \in[0,1]**K, but in the example given in the proof (the \mu_k) it seems not always the case. I think this is an issue because, as mentioned above about line 143, we need to usethat R_k\geq R_\mu_k. You can see that \mu_k do not belong to [0,1]**K first of all because there are negative numbers (-\epsilon_1). Second, isn't there an issue with the constant c? In the end the constant c is put to 4 but it seems that, in some cases, the constant should be smaller than 1 by construction. Indeed we want \epsilon_k \leq 1 so that the \mu_k belong to [0,1]**K. So if the strategy \pi is such that the regret R_k on arm k is linear (R_k=n), as it could happen if the strategy never pulls k, then we would have \epsilon_k=c. So c\leq 1. But this creates a problem with the end of the proof where we need c > 2.In the passage (a) of line 150 why is there a T_k(n) appearing in the square root? It is not obvious from the mentioned standard entropy inequality.equation line 157: on the first equality the \epsilon_k should be \epsilon_1 as defined in \mu_1. This seems to create problem for the following reasoning in the same line. In this same line, in the 'last but one' term, I guess it should be already R_k instead of R_\mu_kMore generally, shouldn't the constant c depend on \pi? Because otherwise the Equation 2 is proving that any possible strategy is pulling all the arms at least linearly! (when c=4, as used in the proof). -- Other possible typos in the rest of paper: line 269: In the second part, at the end of the line: where is it proven that \Delta_i\E\tau_i\leq \sqrt{n_i*}. Is it useful? I see the part with n_i but not with n_i*.line 289: a should be \alpha?line (d) of Corollary 6. Shouldn't it be \alpha+1 and \alpha+1/2 in the denominator? Then the 'proof by graph' becomes even more fishy if their is a typo in the expression (though I guess it is not changing much).line 305: (b) it should be \Delta_i\geq 4*sqrt(n_i*) right?line 311: \tilde \Delta_l is not defined for l=0.


Robust PCA with compressed data
The robust principal component analysis (RPCA) problem seeks to separate lowrank trends from sparse outliers within a data matrix, that is, to approximate a n⇥d matrix D as the sum of a low-rank matrix L and a sparse matrix S. We examine the robust principal component analysis (RPCA) problem under data compression, where the data Y is approximately given by (L+S)·C, that is, a low-rank + sparse data matrix that has been compressed to size n ⇥ m (with m substantially smaller than the original dimension d) via multiplication with a compression matrix C. We give a convex program for recovering the sparse component S along with the compressed low-rank component L · C, along with upper bounds on the error of this reconstruction that scales naturally with the compression dimension m and coincides with existing results for the uncompressed setting m = d. Our results can also handle error introduced through additive noise or through missing data. The scaling of dimension, compression, and signal complexity in our theoretical results is verified empirically through simulations, and we also apply our method to a data set measuring chlorine concentration across a network of sensors to test its performance in practice.

This paper proposes a convex robust PCA method that differs from previous work in that it is designed to be applied to compressed data (i.e. to a lower-dimensional linear transformation of the data).The contributions include the problem statement, its formulation as a convex program, a theoretical approximation bound, specializations for specific assumptions on the noise and compression matrix, empirical results on small synthetic data illustrating the rates of convergence are in accordance with theory, and one small synthetic result providing circumstantial evidence that the approach allowing for sparse perturbations appears beneficial to standard low-rank matrix factorization on the data at hand.The work presented includes non-trivial results that require a high degree of sophistication. The method itself is new and possibly important in a number of practical applications.On the other hand, none of the contributions are breaking new ground. E.g. the problem statement and formulation are straightforward given the literature on compressed sensing and convex approaches multi-task learning and multi-variate regression. The theory, while by no means trivial, contains results of a similar kind and relies on existing proof techniques presented in earlier work (e.g. [1]).The experiments raise questions regarding scalability (surely it is possible to solve this optimization problem for larger data sets?). While they do illustrate the rate of convergence is similar to what the bounds predict, it would have been useful to also see to what extent the constants in the bound are tight (e.g. by plotting the actual bounds on top of the same figures).


Where are they looking?
Humans have the remarkable ability to follow the gaze of other people to identify what they are looking at. Following eye gaze, or gaze-following, is an important ability that allows us to understand what other people are thinking, the actions they are performing, and even predict what they might do next. Despite the importance of this topic, this problem has only been studied in limited scenarios within the computer vision community. In this paper, we propose a deep neural networkbased approach for gaze-following and a new benchmark dataset, GazeFollow, for thorough evaluation. Given an image and the location of a head, our approach follows the gaze of the person and identifies the object being looked at. Our deep network is able to discover how to extract head pose and gaze orientation, and to select objects in the scene that are in the predicted line of sight and likely to be looked at (such as televisions, balls and food). The quantitative evaluation shows that our approach produces reliable results, even when viewing only the back of the head. While our method outperforms several baseline approaches, we are still far from reaching human performance on this task. Overall, we believe that gazefollowing is a challenging and important problem that deserves more attention from the community.

The authors propose a vision method to estimate where within an image a pictured head is looking.Given the image and the cropped out head, the system returns a saliency map consisting of confidence ratings on grid cells saying how likely that position is to be the subject of that head (person)'s gaze.The technique uses CNN with two pathways, one for the head/gaze and one for the full image/saliency of the scene.The authors also contribute a dataset pulling relevant images from SUN/COCO/Actions40/PASCAL/Imagenet, and have it annotated for gaze points by Mech Turk workers.This dataset contains in total 36K people. The method is compared to a few reasonable baselines that represent alternative approaches one might implement and sanity checks.(To my knowledge existing methods in this space are more restrictive than the proposed method, so they are not easily comparable without taking subsets of the test data that satisfy those methods' requirements.)Quantitative results show the promise, and the qualitative analysis and figures are interesting and well composed.The paper tackles the very practical and yet under-explored problem.The solution is simple but effective.It has low technical novelty; it is largely a straightforward application of CNNs with simple input descriptors for the head and scene.There is the detail about the shifted grids that was important to results.  Nonetheless, the dataset collection effort and systematic results (including reasonable baselines) make it valuable.Others are likely to build upon this work.Paper clarity is very good.It is an enjoyable paper to read with many good illustrative figures.  Unclear points/ questions:*More detail could be given about the annotation process and instructions etc.For example, how were the annotators told to decide on a gaze point?Is it literally the point of fixation as they perceive it from that person's line of sight?Or the center of the object they are fixating on?What rules were used to decide what constitutes a poor annotation for discarding?*How many images are in the labeled dataset, after pruning?There are 36K people instances.*It is good that the results compare against human agreement on the test set, where 10 people labeled each test image.Still, why not get multiple gaze annotations for the training data as well, to find a consensus for the ground truth?* In Sec 3.3 this sentence is unclear "Since we only supervise...subproblems."*Text can explain why the uniform distribution of gazes was sought in the test set.Presumably this is to avoid bias.*The SVM baseline description (Lines 333-335) is not clear.Related work:*This sentence about related work is unclear in the intro:"Only [17] tackles the unrestricted..."What is meant by "pre-built components" and why can't it handle people looking away from the camera?If it only handles people looking at the camera, then how is it doing gaze estimation at all?*The paper notes that [7] uses an eye tracker for gaze in egocentric video.The same authors have a more recent paper that removes the need for the gaze tracker: Learning to Predict Gaze in Egocentric Video, Yin Li, Alireza Fathi, James M. Rehg, ICCV 2013*The proposed work seems also related to the interactee prediction work of Chen & Grauman (ACCV 2014).It too tries to produce a "saliency map form the point of view of the person inside the picture (Line 75)", uses similar features (but not CNNs), and produces a multi-modal distribution as its estimate (but using a mixture density neural network instead of classifier).The proposed work is still distinct, mostly because it cares strictly about gaze and treats the interacting/gazed upon objects only implicitly, but I think that work can be cited and the differences explained.Perhaps the insight of the ACCV work about representing the scale and position of the gazed upon object would be relevant to improve this system as well.Predicting the Location of "Interactees" in Novel Human-Object Interactions.C-Y. Chen and K. Grauman.In Proceedings of the Asian Conference on Computer Vision (ACCV), Singapore, Nov 2014.  Results:*The results would be more well-rounded by including failure cases and discussion on failure modes.*I can guess why the Places-CNN and Imagenet-CNN were used for the saliency and gaze components, respectively.But this motivation can be written in the text.(Sec 3.3)I am also curious if it matters in practice to have scenes/objects for the two, or whether results would be similarly if e.g. both were initialized with Imagenet or Places.* A possible enhanced baseline over Fixed Bias: take a subset of training images for which there is a similar distribution of heads.And how is "same head location" determined for this baseline?  Typo: Line 353 " the in the"


Deep Knowledge Tracing
Knowledge tracing-where a machine models the knowledge of a student as they interact with coursework-is a well established problem in computer supported education. Though effectively modeling student knowledge would have high educational impact, the task has many inherent challenges. In this paper we explore the utility of using Recurrent Neural Networks (RNNs) to model student learning. The RNN family of models have important advantages over previous methods in that they do not require the explicit encoding of human domain knowledge, and can capture more complex representations of student knowledge. Using neural networks results in substantial improvements in prediction performance on a range of knowledge tracing datasets. Moreover the learned model can be used for intelligent curriculum design and allows straightforward interpretation and discovery of structure in student tasks. These results suggest a promising new line of research for knowledge tracing and an exemplary application task for RNNs.

This article proposes to model student learning in the context of MOOC using RNN. The system is evaluated according to its ability to predict the exercices on which a given student will succeed or not. The authors present a LSTM RNN architecture to tackle this problem and demonstrate the performance improvement over BKT standard method.This article is an well documented application paper on knowledge tracing, where LSTM RNN seems to be a great advance. Dataset & baselines are relevant and the authors give their code in supplementary material: this effort should be remarked. However, it would have been necessary to add a small README to make the code usable easily.The organization of the paper should be cleaned: the definition of knowledge tracing in subsection 1.1 is alone and it should not be part of the introduction. On top of that, section 4 is strange: it should be split and integrated in the introduction and the related work. I do not understand the motivation is using AUC as evaluation metrics: why not relying on accuracy? This choice is not motivated and does not seem to be the reference in knowledge tracing (even if I am not an expert of the field and I saw that Fig 3 presents also accuracy).Sec 6.2 (and supplementary material) is interesting. Could you imagine a way to build a quantitative evaluation of this task?In conclusion, I found this article interesting and well written. The breakthrough in performance should justify a publication in NIPS.