Preconditioned Spectral Descent for Deep Learning
Deep learning presents notorious computational challenges. These challenges include, but are not limited to, the non-convexity of learning objectives and estimating the quantities needed for optimization algorithms, such as gradients. While we do not address the non-convexity, we present an optimization solution that exploits the so far unused "geometry" in the objective function in order to best make use of the estimated gradients. Previous work attempted similar goals with preconditioned methods in the Euclidean space, such as L-BFGS, RMSprop, and ADAgrad. In stark contrast, our approach combines a non-Euclidean gradient method with preconditioning. We provide evidence that this combination more accurately captures the geometry of the objective function compared to prior work. We theoretically formalize our arguments and derive novel preconditioned non-Euclidean algorithms. The results are promising in both computational time and quality when applied to Restricted Boltzmann Machines, Feedforward Neural Nets, and Convolutional Neural Nets.

I find the explanation of the ADAspectral and RMSspectral interesting. A few comments that I have are: (1) Readability for wider audience:   - It might be useful (maybe in the supplementary material) to provide a  more verbose description of how one gets the update rule for either of  the methods. Or even better, code ! Most readers will first want to know  how hard it is to implement and how well it works before trying to  understand the math. From this perspective, algorithm 1 looks cryptic  with the #-operator and b-operator in there.    - I know it is said, but make it more clearer from the beginning that these  approach amounts to having a different update rule for the weights  (which are matrices) compared to biases. And that for the weights it  involves an SVD (or approximate SVD) over the gradient of these weights.  I think the paper will have a much bigger impact if it is really easy to  follow and clear from the start what the end result will be, even for one  that is not used to these things. (2) Intuitively one way of understanding this algorithm is the following(please correct me if I'm wrong). Say you set D to I (identity). Than youare simply using the sign of the gradient for the biases (multiply this bythe norm of the gradient), and projectingthe gradients of the weights on a sphere of radius equal to the largestsingular value. Now if D is different from I, it is almost like you rely onD completely to tell how much more you should move along an axis vsanother. Again for the biases this is simpler to see than for the weights. * First of all, it would be interesting to compare this with something likeRprop. Rprop basically keeps only the sign of the gradient and ignores itsmagnitude. The RMSspectral seems to project the gradients on some spherelike Rprop (just that you do a rescaling of the gradients with D beforethat and the radius of the sphere is not the learning rate, but the norm ofthe gradient), butthen to change the shape of the sphere based on D. * Secondly it would be interesting to know how vital is for D to be a goodapproximation of the curvature. E.g. it feels that if I make D to be I,I would get an algorithm that probably is much slower than SGD !? Is thisbecause now I don't have a tight bound on F(y). Is there some underlyingassumption on D in the derivation of the algorithm that can be easilymade visible (e.g. the bound becomes tighter if D becomes closer to thecurvature?). (3) I find it hard to compare the results the authors have with other resultsin the literature because they present the results in terms of normalizedtime (instead of actual time). It would have been useful to run the sameexperiment as the one of some other already published paper onoptimization and present the results in a similar fashion (e.g. the deepautoencoder on MNIST or something else).Minor things: * tiny spelling mistakes (e.g. line 403: epcohs)* figure 3, what does normalized time mean?


Efficient and Robust Automated Machine Learning
The success of machine learning in a broad range of applications has led to an ever-growing demand for machine learning systems that can be used off the shelf by non-experts. To be effective in practice, such systems need to automatically choose a good algorithm and feature preprocessing steps for a new dataset at hand, and also set their respective hyperparameters. Recent work has started to tackle this automated machine learning (AutoML) problem with the help of efficient Bayesian optimization methods. Building on this, we introduce a robust new AutoML system based on scikit-learn (using 15 classifiers, 14 feature preprocessing methods, and 4 data preprocessing methods, giving rise to a structured hypothesis space with 110 hyperparameters). This system, which we dub AUTO-SKLEARN, improves on existing AutoML methods by automatically taking into account past performance on similar datasets, and by constructing ensembles from the models evaluated during the optimization. Our system won the first phase of the ongoing ChaLearn AutoML challenge, and our comprehensive analysis on over 100 diverse datasets shows that it substantially outperforms the previous state of the art in AutoML. We also demonstrate the performance gains due to each of our contributions and derive insights into the effectiveness of the individual components of AUTO-SKLEARN.
In practice, the budget b would comprise computational resources, such as CPU and/or wallclock time and memory usage. This problem definition reflects the setting of the ongoing ChaLearn AutoML challenge [1]. The AutoML system we describe here won the first phase of that challenge.
Here, we follow and extend the AutoML approach first introduced by AUTO-WEKA [2] (see http://automl.org). At its core, this approach combines a highly parametric machine learning framework F with a Bayesian optimization [3] method for instantiating F well for a given dataset.
The contribution of this paper is to extend this AutoML approach in various ways that considerably improve its efficiency and robustness, based on principles that apply to a wide range of machine learning frameworks (such as those used by the machine learning service providers mentioned above). First, following successful previous work for low dimensional optimization problems [4,5,6], we reason across datasets to identify instantiations of machine learning frameworks that perform well on a new dataset and warmstart Bayesian optimization with them (Section 3.1). Second, we automatically construct ensembles of the models considered by Bayesian optimization (Section 3.2). Third, we carefully design a highly parameterized machine learning framework from high-performing classifiers and preprocessors implemented in the popular machine learning framework scikit-learn [7] (Section 4). Finally, we perform an extensive empirical analysis using a diverse collection of datasets to demonstrate that the resulting AUTO-SKLEARN system outperforms previous state-of-the-art AutoML methods (Section 5), to show that each of our contributions leads to substantial performance improvements (Section 6), and to gain insights into the performance of the individual classifiers and preprocessors used in AUTO-SKLEARN (Section 7).

SUMMARYThe authors introduce a new automated machine learning (autoML) system for classification, as well as associated technical advances that improve upon its baseline performance.The new system is referred to as auto-sklearn, since it is built upon the scikit-learn library of machine learning algorithms.The system uses 16 classification algorithms, 11 preprocessing methods, and 3 data preprocessing methods.The baseline system involves optimizing over algorithms and hyperparameters for each algorithm, to find a choice that minimizes the cross-validation error.This is accompanied by two further contributions. First, the authors experiment with meta-learning to assist the optimization; this involves extracting meta-features from a large collection of data sets (140, in this case), and characterizing the types of data sets on which each algorithm performs well.Second, the authors experiment with automated construction of ensembles to further improve the classification performance; this is done via an efficient ensemble selection method combining intermediate points evaluated during the optimization.The performance of the baseline system, as well as the augmented system with and without meta-learning and ensembling, is evaluated on 140 diverse data sets. The baseline system tends to provide better performance compared to two existing autoML systems: Auto-WEKA and hyperopt-sklearn. On top of this, employing meta-learning significantly improves performance beyond the baseline, and using ensembling in addition improves it still further.The authors also provide a detailed analysis of the performance of the 16 individual algorithms, providing insight into the relative merits of various standard classifiers. COMMENTSThis work represents a significant effort in terms of engineering and computation, for which authors are to be commended.The article is clearly written and easy to read. The motivation, formulation of the problem, description of the proposed method, description of experiments, and presentation of results are all excellent. The proposed system demonstrates an impressive advance in autoML performance. The use of meta-learning, in particular, is very interesting.The observation that ensemble selection can be used to efficiently combine intermediate points from the optimization is also nice.My criticisms are relatively minor.1. A possible criticism is that this work largely represents an engineering and computational effort, with few significant advances in terms of novel techniques or theory. However, I believe this is outweighed by the scale of the effort, the strength of the performance demonstrated, and the potential utility of this tool.2. Line 92-93: If the number of algorithms m is large, could this lead to overfitting, even though we are optimizing with respect to cross-validation error?3. Do the authors have any insights into how best to choose the number and type of algorithms to include in the system? The current system seems to be based simply on what is available in scikit-learn. Perhaps in future work, it would be interesting to know if there is a principled way to choose the composition of algorithms to be included, beyond the general suggestion that it helps to have a diversity of algorithms.4. The meta-learning approach is appealing. However, no explanation is given for why the authors chose the particular approach that they did. The approach chosen is roughly analogous to k-nearest neighbors, but one can imagine many other possibilities involving other classifiers or regression algorithms. Are there reasons to prefer the chosen approach over other approaches? 5. Perhaps in future work, it would be interesting if the meta-learning aspect of the system could be formalized via a mathematical objective function that could be optimized in order to improve the performance of the overall system.6. Line 150: Minor point --- I don't get how this is Bayesian... I would think a Bayesian approach would involve something like putting a prior on the algorithms/hyperparameters, and sampling from the posterior.7. Tables 4 and 5: Minor point --- it might be helpful to move the auto-sklearn column to the front or back, in order to highlight it. Or maybe to sort the columns alphabetically or by aggregate performance.


Time-Sensitive Recommendation From Recurrent User Activities
By making personalized suggestions, a recommender system is playing a crucial role in improving the engagement of users in modern web-services. However, most recommendation algorithms do not explicitly take into account the temporal behavior and the recurrent activities of users. Two central but less explored questions are how to recommend the most desirable item at the right moment, and how to predict the next returning time of a user to a service. To address these questions, we propose a novel framework which connects self-exciting point processes and low-rank models to capture the recurrent temporal patterns in a large collection of user-item consumption pairs. We show that the parameters of the model can be estimated via a convex optimization, and furthermore, we develop an efficient algorithm that maintains O(1/ ) convergence rate, scales up to problems with millions of user-item pairs and hundreds of millions of temporal events. Compared to other state-of-the-arts in both synthetic and real datasets, our model achieves superb predictive performance in the two time-sensitive recommendation tasks. Finally, we point out that our formulation can incorporate other extra context information of users, such as profile, textual and spatial features.

Paper proposes a new method for time-sensitive recommendations based on user activities. It is different from existing methods because it address the problem of time-varying user preferences. The paper also addresses prediction of the next returning time of a user. Learning is performed using an efficient optimization algorithm proposed by authors. Experimental section shows results on one synthetic data set in order to show that learning is efficient on large scale data and on two real data sets of modest size.First, is a problem of time-varying user preferences really that spread. It would be great to analyze the real data sets from experiments in order to make this claim. If so, is it varying mostly because new popular songs come out? Or there some other pattern?What is the motivation for predicting next returning time for user? Could you please motivate it before you claim it as contribution of a paper? How could a web-company use this prediction and for what?I believe there are some simple baselines for predicting the next returning time that were not tested: 1) the last time delay 2) the most occurred time delay. Also, I think that predicting the actual returning time of the user is too challenging. Instead, I would focus on a more simpler task that is easier to predict. For example predicting is user will have short delay (1-5 days), medium delay (5-14 days), long delay (14-33 days) or very long delay (>33 days) before he uses our service again.Experimental section: It was hard for me to get a sense of how significant the performance lift is compared to SVD. Is it groundbreaking (order of magnitude) or modest? Would the user feel the difference? How would these recommendations compare to recommending popular items? In many cases it is a hard baseline to beat, especially if popular items are re-calculated every day.Small comments:1) Typo in last paragraph on page 3: "and is abel to capture", abel -> able2) In first paragraph of section 2 you start mentioning retweets and link creation in social networks without any references. The change of context was confusing.. best to rephrase.


An Active Learning Framework using Sparse-Graph Codes for Sparse Polynomials and Graph Sketching
Let f : {−1, 1} n → R be an n-variate polynomial consisting of 2 n monomials, in which only s 2 n coefficients are non-zero. The goal is to learn the polynomial by querying the values of f . We introduce an active learning framework that is associated with a low query cost and computational runtime. The significant savings are enabled by leveraging sampling strategies based on modern coding theory, specifically, the design and analysis of sparse-graph codes, such as Low-Density-Parity-Check (LDPC) codes, which represent the state-of-the-art of modern packet communications. More significantly, we show how this design perspective leads to exciting, and to the best of our knowledge, largely unexplored intellectual connections between learning and coding. The key is to relax the worst-case assumption with an ensemble-average setting, where the polynomial is assumed to be drawn uniformly at random from the ensemble of all polynomials (of a given size n and sparsity s). Our framework succeeds with high probability with respect to the polynomial ensemble with sparsity up to s = O(2 δn ) for any δ ∈ (0, 1), where f is exactly learned using O(ns) queries in time O(ns log s), even if the queries are perturbed by Gaussian noise. We further apply the proposed framework to graph sketching, which is the problem of inferring sparse graphs by querying graph cuts. By writing the cut function as a polynomial and exploiting the graph structure, we propose a sketching algorithm to learn the an arbitrary n-node unknown graph using only few cut queries, which scales almost linearly in the number of edges and sub-linearly in the graph size n. Experiments on real datasets show significant reductions in the runtime and query complexity compared with competitive schemes.

The paper studies the problem of actively learning a function f:{-1,1}^n -> R which is a sum of at most s parity functions or in other words its Fourier spectrum is sparse. The algorithm is allowed to query the value of the function at any desired point z in F2^n.       The main result is: For any function whose parity functions are randomly uniformly drawn from the set of all parities and whose coefficients are drawn from a distribution on a bounded set, the algorithm runs in time O(ns logs) with O(ns) queries and learns the function. This is also possible when the queries are noisy.   Another special case of the result is when the parities have at most d variables in them. In this case, the algorithm has O(sd logs logn) query complexity and O(ds log2s logn) run time. Further, this is applied to the problem of learning hypergraphs from cuts (graph sketching) and when the hyperedges are of constant size, the algorithm learns the graph in O*(r) time where O*() involves some poly log terms in n and r. This is strictly better than any algorithm known all of which needs poly in n running time. The results have been compared to another set of algorithms developed for the case of sparse function learning from random samples. Simulations on real data sets have been provided that demonstrate the gain from other algorithms. Main Ideas:Actually, in finding coefficients of parities and in also finding the parities, this work designs queries in such a way that a "sparse parity-check like equations" akin to decoding LDPC codes in coding theory is formed and peeling decoders are used to find the solution. In detail the work uses the following two ideas:Idea 1: Fourier expansion of f is a Walsh-Hadamard Transform with the coefficients of parities serving as the WHT coefficients. Now if one takes the s- point WHT (WHT is also its inverse) of a bunch of queries evaluated on a hyperplane H in F2n of dimension b = O(log s),then each of the coefficients of the s-point WHT is a very sparse linear combination of only a subset of coefficients of parities. The s-point WHT of the queries and the coefficients of the actual parities form a sparse bipartite graph where peeling decoding is employed. However, this can reveal only the coefficients of the parities but not the parities themselves. For that authors employ the following ideaIdea 2: Each of the s-point WHT of the queries is formed by linearly combining +1s and -1s and the sign depends only on the shift of the hyperplane H from origin and the parity.So multiple hyperplanes are used to create different +-1 linear combinations. Now these can be used at the time of peeling, to relate the actual parity and the sign of the s-point WHTs for different hyperplanes.So O(n) hyperplanes are necessary in general. But if the parities are of bounded degree d, then clearly clever design of the hyperplane shifts can result in another compresses sensing problem with bi-partite graphs and peeling decoding which will reduce the number of hyper planes requires to O(d log n). Strengths: The results for bounded degree parities and its subsequent application graph sketching are novel and they almost match the information theoretic lower bound of O*(r) query complexity for the graph sketching problem. The simulations are very interesting and its comparison with previous algorithm for the random examples case illustrates its utility.Weaknesses: 1) This is my most important comment/suggestion:In section 1.2 and in several places, the solution of this work is compared with ref16, ref21 and ref9. But those algorithms are for random samples with no query access. However, the Query access model is a lot more powerful. So I think the authors could try comparing it with Kushilevitz-Mansour algorithm (refer to Thm 3.37 in the Book:Analysis of Boolean functions by O'Donnell). Kushilevitz-Mansour algorithm (with some very simple modifications to the case when f:{-1,1}n-> R) can be made to run in time poly(n,s) and it also designs queries adaptively. Closely, looking at it, it seems like it has very similar queries and run time complexities although I could be wrong. However, the bounded degree case seems novel and the resulting graph sketching application seems novel. But even in this case, the algorithm given in page 85, of the same book, could be modified to start with buckets of size nd instead of size 2n as in conventional Kushilevitz-Mansour algorithm and I think that would give the O(d log n) scaling that accompanies the results in the paper.However, I am not very sure about the second suggestion. Authors could clarify this point.2) Minor comments:a) Choices for matrices Mc in page (4) is not mentioned in the main paper. It seems like it should not matter much. But I am not sure what role it plays in the peeling decoding analysis.b) Proposition 1 "...where each group generates P a query set.. "-> 'a' should be removed. c) g_{\hat{k}} in algorithm 1 is not defined in the main paper. d) Last but the second line before Definition 3: "that is is ..." -> only one 'is'. e) \mathbf{H} is defined using MATLAB notation of concatenating matrices row wise. Could clarify that notation somewhere. f) Section 3 Line 4 - "..cut xcan be..." - > needs to change.


Bandits with Unobserved Confounders: A Causal Approach
The Multi-Armed Bandit problem constitutes an archetypal setting for sequential decision-making, permeating multiple domains including engineering, business, and medicine. One of the hallmarks of a bandit setting is the agent's capacity to explore its environment through active intervention, which contrasts with the ability to collect passive data by estimating associational relationships between actions and payouts. The existence of unobserved confounders, namely unmeasured variables affecting both the action and the outcome variables, implies that these two data-collection modes will in general not coincide. In this paper, we show that formalizing this distinction has conceptual and algorithmic implications to the bandit setting. The current generation of bandit algorithms implicitly try to maximize rewards based on estimation of the experimental distribution, which we show is not always the best strategy to pursue. Indeed, to achieve low regret in certain realistic classes of bandit problems (namely, in the face of unobserved confounders), both experimental and observational quantities are required by the rational agent. After this realization, we propose an optimization metric (employing both experimental and observational distributions) that bandit agents should pursue, and illustrate its benefits over traditional algorithms.

The paper "Bandits with unobs. confounders: a causal approach" addresses the problem of bandit learning. It is assumed that in the observational setting, the player's decision is influenced by some unobserved context. If we randomize the player's decision, however, this intention is lost. The key idea is now that, using the available data from both scenarios, one can infer whether one should overrule the player's intention. Ultimately, this leads to the following strategy: observe the player's intention and then decide whether he should act accordingly or pull the other arm.  I like the problem the paper addresses and the way it approaches it. Although I would have liked to do so, however, I cannot strongly recommend the acceptance of this paper due to1) it contains almost no material beyond an idea 2) limitations to bivariate variables? 3) method is not properly compared to contextual bandits, experiments available only on two artificial toy data sets I definitely encourage the authors to continue this line of research and provide a more thorough discussion of this problem.- the key idea seems to be (7), which computes a counterfactual statement from observational and interventional data. unfortunately, this is already known (e.g. "What Counterfactuals Can Be Tested" Shpitser, Pearl, p.353).- also, i am not sure how the idea generalizes to variables with more than 2 states (i.e., #arms and/or pay-off). it is known that in general, counterfactuals cannot be inferred from obs. and int. data. This seems to be a major limitation that is not discussed properly.- the assumption that the intention of the player can be measured before he pulls an arm is quite strong. but if it can, this problem is "just" a contextual bandit (e.g. use TS^Z). It remained unclear to me, why in the simulations, the proposed method got access to the observational data, while the TS^Z approach did not (if it did, it should be at least as good!?). I think, one could rewrite the whole motivation of the paper: consider a situation, in which one first observes a _different_ player (obs. data) playing a bandit and then oneself sits down in order to gather the interventional data. this requires the assumption that all players develop the same intention. but then it seems to boil down to a contextual bandit with a context that is partially hidden for some data points. Is there any literature on this problem? - the first concrete idea (as far as I can tell) is in the last paragraph on page 6 (for a conference paper, the introduction is too long)- there is no code available- there are only toy examples- in many places the paper is badly written/unclear. i only give few examples, there are more * bottom page 2, there are two machines (sometimes two arms), but D indicates whether "the" machine is blinking * Y = {0,1} doesn't make sense if Y is a rand.var. * the authors confuse E and P many times. E(Y) = P(Y=1) if Y takes values in {0,1} and change notation. * alg. 1 contains many unclear statements, e.g. beta and f are not defined properly, what is x' etc. * the writing f(x,a) for all a = x, is not very sensible, bottom p. 6 * the abstract is a bit unclear "passive data through random sampling"- maybe focus more on the example?- the colors in fig. 1 are indistinguishable minor - greendy casino - use \eqref - shouldn't (8) be <=> ? - is great than - citations are very inhomogeneous - random experiments do not control for but eliminate the influence of hidden confounders. - what are the grey lines in Fig. 2? - I would not regard Def. 3.3. as a definition


Learning to Segment Object Candidates
Recent object detection systems rely on two critical steps: (1) a set of object proposals is predicted as efficiently as possible, and (2) this set of candidate proposals is then passed to an object classifier. Such approaches have been shown they can be fast, while achieving the state of the art in detection performance. In this paper, we propose a new way to generate object proposals, introducing an approach based on a discriminative convolutional network. Our model is trained jointly with two objectives: given an image patch, the first part of the system outputs a class-agnostic segmentation mask, while the second part of the system outputs the likelihood of the patch being centered on a full object. At test time, the model is efficiently applied on the whole test image and generates a set of segmentation masks, each of them being assigned with a corresponding object likelihood score. We show that our model yields significant improvements over state-of-theart object proposal algorithms. In particular, compared to previous approaches, our model obtains substantially higher object recall using fewer proposals. We also show that our model is able to generalize to unseen categories it has not seen during training. Unlike all previous approaches for generating object masks, we do not rely on edges, superpixels, or any other form of low-level segmentation.

A neural network inputs an image and generates a set of proposal segmentation masks and an objectness score for each one. It is trained on COCO dataset given the segmentation masks.This is great that the model works well on object categories that it has not seen in training.Why do we want to solve this problem? Reducing the number of candidates can speed up the object detection. I am not sure it can improve the average precision of detection as the total recall for previous approaches like "selective search" is already really high. I think that would be great to apply an off-the-shelf object detection in the output of this method to see the improvement in the speed vs. accuracy. Moreover, this method produces segmentation masks which is great. However, I think it should be compared with the proper segmentation baselines so that we can see the performance.Table 1, Table 2, and Figure 3 show the recall at maximum of 1000 proposals which is not a big number. (Some papers use 2000 candidates for selective search) That would be great to extend it to a larger number say 5000 to see if different approaches converge. For instance, if the recall of this method converges to selective search in 2000 proposals, what is the benefit of this method? I know that this method generates segmentation masks as well, but then, there are quite a few baselines that can be considered to evaluate the segmentation masks. Line 269: that would be great to discuss why downsampling the ground-truth instead of up-sampling the network output degrades the accuracy.The colors on Fig 3 are not easy to read. One can change the pattern of lines (to dots or etc).


M-Best-Diverse Labelings for Submodular Energies and Beyond
We consider the problem of finding M best diverse solutions of energy minimization problems for graphical models. Contrary to the sequential method of Batra et al., which greedily finds one solution after another, we infer all M solutions jointly. It was shown recently that such jointly inferred labelings not only have smaller total energy but also qualitatively outperform the sequentially obtained ones. The only obstacle for using this new technique is the complexity of the corresponding inference problem, since it is considerably slower algorithm than the method of Batra et al. In this work we show that the joint inference of M best diverse solutions can be formulated as a submodular energy minimization if the original MAP-inference problem is submodular, hence fast inference techniques can be used. In addition to the theoretical results we provide practical algorithms that outperform the current state-of-the-art and can be used in both submodular and non-submodular case.

The paper considers the problem of finding best diverse solutions of energy minimisation problems with focus on submodular energy functions. The authors further present an alternative formulation of the problem as an ordering enforcing problem. Furthermore, the proposed methods are compared to other state of the art methods in experiments.Quality. The presented results are interesting and seem to be correct. However, I'm not sure of the implications of Lemma 2. By replacing the original diversity measure by the one in (15), supermodularity of the diversity measure is achieved. What effects does this replacement have? Do the characteristics of the diversity measure change? Furthermore, regarding Corollary 2, it would be interesting which other diversity measures satisfy the given condition. Regarding the experiments, the authors seem to report the best result from all M obtained labelings. Wouldn't it make sense to also report the average quality and the corresponding standard deviation? Clarity. The clarity of the manuscript is good, but could be improved at several points: At several points, e.g. in the appendix, the authors refer to "the algorithms", however it is unclear to me what exactly they refer to. In equations (11) and (15) the authors use the same notation for different quantities. It would be interesting to explicitly state how these two diversity measures are related and what are the implications for the obtained diverse labelings when using the one or the other.Originality. The paper improves results for identifying diverse solutions jointly over a recently submitted paper, especially by approaching runtime/complexity issues. While there is significant overlap, this improvements seem sufficiently interesting and original to me.Significance. The presented results could be interesting for many applications in many different fields.Further comments: * L152: "This implies that in this case" * L184: "The corresponding factor ..." * L196: "compared to" * L 237: "If E is submodular, then" * L 254: "This order can be enforced" * Table 1: State meaning of "M" again. * Provide details on how the cross-validation for selecting lambda is performed. * L404: "the intersection over union quality measure" ... what does this mean? * L430: "much smaller than" * Is there any reasonable approach to learn which diversity measures to combine? * Please carefully check the grammar of the supplement. * several places: "there holds" => "it holds"


Top-k Multiclass SVM
Class ambiguity is typical in image classification problems with a large number of classes. When classes are difficult to discriminate, it makes sense to allow k guesses and evaluate classifiers based on the top-k error instead of the standard zero-one loss. We propose top-k multiclass SVM as a direct method to optimize for top-k performance. Our generalization of the well-known multiclass SVM is based on a tight convex upper bound of the top-k error. We propose a fast optimization scheme based on an efficient projection onto the top-k simplex, which is of its own interest. Experiments on five datasets show consistent improvements in top-k accuracy compared to various baselines.

In many multiclass classification problems with a large number of classes, semantic similarity between classes can typically be observed. This paper proposed an algorithm called top-k SVM to explicitly deal with the learning scenario where you only get a cost when the ground truth is outside the top k proposed classes (e.g. imagenet challenge). After extending the loss function to top-k loss, an efficient optimization procedure is formulated. Experiments on large scale datasets showed the scalability of the algorithm and clear improvement could be observed over compared algorithms.The paper is clearly motivated and easy to follow. And the problem (similar classes in multiclass learning) they work on is an important one that happens a lot in many practical applications.


Generalization in Adaptive Data Analysis and Holdout Reuse *
Overfitting is the bane of data analysts, even when data are plentiful. Formal approaches to understanding this problem focus on statistical inference and generalization of individual analysis procedures. Yet the practice of data analysis is an inherently interactive and adaptive process: new analyses and hypotheses are proposed after seeing the results of previous ones, parameters are tuned on the basis of obtained results, and datasets are shared and reused. An investigation of this gap has recently been initiated by the authors in [7], where we focused on the problem of estimating expectations of adaptively chosen functions. In this paper, we give a simple and practical method for reusing a holdout (or testing) set to validate the accuracy of hypotheses produced by a learning algorithm operating on a training set. Reusing a holdout set adaptively multiple times can easily lead to overfitting to the holdout set itself. We give an algorithm that enables the validation of a large number of adaptively chosen hypotheses, while provably avoiding overfitting. We illustrate the advantages of our algorithm over the standard use of the holdout set via a simple synthetic experiment. We also formalize and address the general problem of data reuse in adaptive data analysis. We show how the differential-privacy based approach given in [7] is applicable much more broadly to adaptive data analysis. We then show that a simple approach based on description length can also be used to give guarantees of statistical validity in adaptive settings. Finally, we demonstrate that these incomparable approaches can be unified via the notion of approximate max-information that we introduce. This, in particular, allows the preservation of statistical validity guarantees even when an analyst adaptively composes algorithms which have guarantees based on either of the two approaches.

# SummaryThe paper considers the setting where an analyst has a training set and a hold-out set from the same distribution. The analyst constructs her methods based on the training set, but also repeatedly performs checks on the hold-out set to make important decisions. In general this repeated use of the hold-out set may lead to overfitting to the hold-out set, but if the analyst only performs her checks using the methods introduced in the paper, overfitting is avoided, as shown by the main results: Thms 9 and 10. This extends earlier results based on differentially private methods.# Positive PointsWhat is exciting about this line of results, is that they do not a priori restrict the class of functions (say, to a class of limited VC dimension) from which the analyst can select her checks. Otherwise, standard uniform concentration inequalities for VC classes would suffice. See lines 226-232 in the paper.An interesting point of the paper is that it goes beyond differentially private methods to also include algorithms whose output can be described by a small number of bits. This generalization is based on a unifying quantity called (approximate) max-information. Unlike notions of algorithmic stability, which are known to be closely related to generalization, this max-information allows composing multiple algorithms in a sequence. (See lines 234-243.)The high-level discussion in the paper is generally well-written. # Negative PointsIn spite of all the positive points mentioned above, I have some significant concerns about Thm 9 (one of the main results), which currently makes me believe that the paper would need a serious revision before it could be accepted.# Main concernsRegarding Thm 9: 1. The theorem makes a deterministic statement "for all i such that  a_i \neq \perp", but whether "a_i \neq \perp" is random, because it  depends on the adaptively chosen functions phi_i, on the randomness  in the data, and on the randomness introduced by the algorithm. 2. The in-probability statement bounds the probability that a_i deviates  significantly from the expectation of phi_i for a given fixed i. But  if the analyst adaptively makes decisions based on the output of the  algorithm, then a_i must be close to the expectation of phi_i for all  i simultaneously. Otherwise the analysis might take a wrong turn  somewhere. 3. I have been unable to verify the claim in lines 119,120 of the  introduction, which I paraphrase as: the number of queries m can be  exponential in n as long as the budget B is at most quadratic in n.  In particular, the result does not appear to deteriorate with m,  which I think is because of point 2 above. And in order for the  result to be useful to the analyst, I would expect that tau would go  to 0 with n, for example as tau ~ 1/sqrt(n). But if we let B be  quadratic in n, then having tau go to 0 will make beta go to  infinity, and hence the result becomes vacuous. So I don't know which  parameters to plug into the theorem. About the experiments: why do you construct a classifier that only uses weights +1 and -1 based on the signs of correlations? Is there a plausible scenario in which someone might do this in practice? # Minor remarksAs acknowledged by the authors in their proof of Thm 23 in the additional material, the extension to algorithms whose output can be described using a small number of bits can also be obtained from a union bound argument. This is hinted at in lines 171-179 of the introduction, but may not be entirely clear from the discussion there. # A potential strengtheningFinally, I would like to point out a possible strengthening of the results, which may or may not be useful.Technically, all results are based on a change of measure from the joint distribution P of a sample and an algorithm's output on that sample to the independent product distribution Q of the two (Thm 4). If P and Q are sufficiently close in terms of "max-information", then we may essentially still treat the sample as fresh even if we have already looked at the output of the algorithm.It might be of interest to observe that, at least for beta = 0, the bound on max-information can be relaxed. To do this, the max-information may be interpreted as the Renyi divergence D_alpha(P||Q) of order alpha = infinity. Renyi divergence is increasing in alpha (see e.g. [1]), so requiring a bound on alpha=infty is the strongest requirement one can impose. Thm 4 then states that we can change measures from P to Q if D_infty(P||Q) is sufficiently small. It is actually possible to change measures under the weaker requirement that D_alpha(P||Q) is small for any alpha > 1. For alpha=1 we recover regular mutual information, which, as the authors point out, is not strong enough to allow a change of measure.The proof is essentially a special case of Lemma 1 of [2], but the relation may be hard to see, so allow me to translate. Let X be the indicator random variable, which is 1 if (S,A(S)) in cal{O} and 0 otherwise. Then read Lemma 1 of [2] with E_P[X] instead of L_P(h,f), E_Q[X] instead of L_Q(h,f), and M = max X = 1 to get: E_P[X] <= ( 2^{D_alpha(P||Q)} * E_Q[X] )^{(alpha-1)/alpha} * M^(1/alpha)which is equivalent to P[(S,A(S)) in cal{O}] <= ( 2^{D_alpha(P||Q)} * Q[(S,A(S)) in cal{O}] )^{(alpha-1)/alpha}As alpha -> infinity, we recover Thm 4, but the result also holds for smaller alpha.1. Van Erven, Harremoes, "Renyi Divergence and Kullback-Leibler Divergence, IEEE Transactions on Information Theory, 2014. 2. Mansour, Mohri, Rostamizadeh, "Multiple Source Adaptation and the Renyi Divergence", UAI 2009.  # Minor issues (typo's etc)Line 114: training set -> hold-out set (I believe)Thm 4: two issues: "k = I_infty^beta(S;A(S)) = k" -> I_infty^beta(S;A(S)) <= k?Thm 8: The sentence "Let Y = ... on input S." can be removed.Lines 314-323: the bounds you mention bound the probability that the empirical estimate is more than tau *away* from the true accuracy.


A Market Framework for Eliciting Private Data
We propose a mechanism for purchasing information from a sequence of participants. The participants may simply hold data points they wish to sell, or may have more sophisticated information; either way, they are incentivized to participate as long as they believe their data points are representative or their information will improve the mechanism's future prediction on a test set. The mechanism, which draws on the principles of prediction markets, has a bounded budget and minimizes generalization error for Bregman divergence loss functions. We then show how to modify this mechanism to preserve the privacy of participants' information: At any given time, the current prices and predictions of the mechanism reveal almost no information about any one participant, yet in total over all participants, information is accurately aggregated.

The authors identify a strategy for compensating agents to disclose private and relevant data. The strategy draws on interesting ideas from the machine learning literature, including the use of the stochastic gradient descent algorithm to set the payment for the data. This allows effective compensation of agents while maintaining a limited budget. The authors also include mechanisms for preserving the privacy of agents, and identification of different "profit maximizing" strategies for agent to select given their confidence in their data. It appears that the substantive contribution isin identifying a mechanism that compensates agents for their data while maintaing a bounded budget.The usefulness of this family of algorithms hinges on timely, available correct information.If "true samples" will arise only in the distant future, then section 2.3 is not viable (see below), section 2.1 does not seem to offer much advantage over traditional betting markets (modulo the generalizability to more complex prediction function).Section 3 does not strike me as addressing a fundamentally serious problem: depending on the type of privacy one is concerned with, there seem to be simpler solutions to keep agent i from learning about agent j: (a) anonymized participation (b) anonymized relations between updates and agents (which agent made which trade is not available to other agents)The really useful contribution here is a generalization of prediction markets to use sophisticated prediction tools, thus allowing for prediction markets to extend far beyond bets on single binary or scalar variables.This is cool, but in practice it seems like it would offer little motivation for participation, unless the market happened to run along-side a constant stream of true samples, such that rewards could be evaluated quickly.(If I won't know what my data is bidding on, how much credit it might get, and when the rewards will come, why should i play?)Section 2.3 (offering to buy data) seems predicated on knowing the loss function of predictions, which means that the correct answers are known.In that case, what's even the point of bidding on data?If the answers are not known, then how does one come up with the marginal gain from a given data point?Seems to be a pretty fundamental catch-22.There are a few possible ways out of this, but they all seem to open the system up to being gamed: - Use past true samples to evaluate the worth of new training data.However, then one can simply offer lots of training data to match known past samples, and reap the rewards by generating fake data to overfit past data. - Use some prediction divergence measure to evaluate the utility of new data (i.e., all data that changes my predictions is good).This can obviously be gamed by generating crazy data to make predictions veer wildly offtrack. Thus, it seems that the only way to set up a cost function is in light of *future* true samples, and this seems to be a serious limitation insofar as the future is distant.It absolutely precludes the kind of "selling" scheme in section 2.3


Learning Stationary Time Series using Gaussian Processes with Nonparametric Kernels
We introduce the Gaussian Process Convolution Model (GPCM), a two-stage nonparametric generative procedure to model stationary signals as the convolution between a continuous-time white-noise process and a continuous-time linear filter drawn from Gaussian process. The GPCM is a continuous-time nonparametricwindow moving average process and, conditionally, is itself a Gaussian process with a nonparametric kernel defined in a probabilistic fashion. The generative model can be equivalently considered in the frequency domain, where the power spectral density of the signal is specified using a Gaussian process. One of the main contributions of the paper is to develop a novel variational freeenergy approach based on inter-domain inducing variables that efficiently learns the continuous-time linear filter and infers the driving white-noise process. In turn, this scheme provides closed-form probabilistic estimates of the covariance kernel and the noise-free signal both in denoising and prediction scenarios. Additionally, the variational inference procedure provides closed-form expressions for the approximate posterior of the spectral density given the observed data, leading to new Bayesian nonparametric approaches to spectrum estimation. The proposed GPCM is validated using synthetic and real-world signals.

The paper is concerned with the important problem of analyzing impulse responses of LTI systems with GPs. I feel that the paper is nice, but it does not really project the methods to existing approaches which would be an important thing to do. This is a classical problem anyhow. Some comments:- In Section 2 the authors talk about "convergence" without any reference to the mode of convergence or references to literature. Certainly the convergence properties of the integral (1) are known, e.g., in suitable Schwartz spaces because Wiener alone wrote tens of books on harmonic analysis.- Based on the text I would guess that the authors wish to talk about mean square convergence, which is related to Ito's work (stochastic calculus) and where the integral (1) needs to be defined as an Ito integral f(t) = int h(t - tau) dW(tau) which is how it is classically done (see, e.g., the refs. below). - The problem considered here is called "system identification" (cf. google), and it certainly is not a new problem. This problem is classically treated in the framework of "Kalman filtering", which is "GP regression for time series" and works in the Ito calculus / state-space formalism. Nice classic textbooks on the core subject are, e.g., the following:"A. H. Jazwinski (1970). Stochastic processes and filtering theory." "P. Maybeck (1979). Stochastic models, estimation, and control. Volumes 1&2."- The basic classical method is indeed to use a parametric model for the impulse response h(t) instead of a non-parametric model. Typically this impulse response is constructed in a linear state-space form which is then parametrized suitably. State-space methods such as Kalman filtering based approaches can then be used for parameter estimation directly although the resulting model can be treated with other types of methods as well. To clarify: the Kalman filter can be used to evaluate the marginal likelihood using the prediction error decomposition and conventional parameter inference can be implemented on top of it.- It would be beneficial to compare at least to a very simple parametric model to the impulse response---or at least discuss them. You could, for example, use an all-pole type of transfer function modelH(s) = b0 / (a0 + a1 s + a2 s^2)where the parameters a0,a1,a2,b0 are the free parameters. The inverse Laplace transform of this then implicitly defines the impulse response although it is easier to just convert this into state space form and use a Kalman filter type of method to estimate b0,a0,a1,a2. A more elaborate model would be a higher order rational function model.- One point is that DSP books such as Oppenheim et al. does not treat the above kind of models, but you need to look into (oldish) control engineering and system identification literature to find them---although some newer literature might exist. Just to point out that using the classical state-space/Ito approach above, the non-uniform sampling is *not* a problem at all even though from Oppenheim et al it might seem to be.- However, system identification problems are indeed classically parametric and it is a difference to here and the novelty of the article. Furthermore, the above approach for system identification has number of difficulties, for example, it is quite hard to enforce the transfer function to correspond to a stable system. Thus new methods are welcome. In any case, it is not an excuse to ignore the existing literature, you should at least discuss them.- The approach itself is very simple which is nice. It also points out many connections to inducing point methods which is nice.- The weakness in the experiment section is also the lack of comparison to classical approaches.


Nonparametric von Mises Estimators for Entropies, Divergences and Mutual Informations
We propose and analyse estimators for statistical functionals of one or more distributions under nonparametric assumptions. Our estimators are derived from the von Mises expansion and are based on the theory of influence functions, which appear in the semiparametric statistics literature. We show that estimators based either on data-splitting or a leave-one-out technique enjoy fast rates of convergence and other favorable theoretical properties. We apply this framework to derive estimators for several popular information theoretic quantities, and via empirical evaluation, show the advantage of this approach over existing estimators.

Summary of paper: This paper presents a leave-one-out (LOO) estimator for a functional of a single distribution and second LOO estimator for a functional of two distributions. The authors present analysis for these LOO estimators as well as for data split (DS) estimators. Lastly, the paper presents experiments on synthetic datasets and an benchmark dataset. The authors have many goals for this paper from introducing new ideas to the machine learning community to proofs and experiments. It feels as though there could be two papers here: one for functionals of a single distribution and a second for functionals of two distributions. By splitting this into two papers, the authors would have more space in the main paper to add additional exposition detailing the purpose of each idea introduced, instead of relying so heavily on the supplemental material. This paper is a challenging read. The authors have several moments of clarification that are a welcome break from the deluge of technical functional analytic details. For example, the last sentence of the first paragraph does an excellent job summarizing the majority of the paper. Further examples of helpful clarification are the discussions following the list of contributions (starting at line 75) and Theorem 8 (starting at line 308). In contrast, at the top of the second page, the authors state that they will "introduce the calculus of influence functions to the machine learning community." While the authors do provide a technical definition for influence functions, they do not provide approachable exposition on influence functions, explaining what they are, what they can be used for, or why one may use them. Despite reading the paper several times, I had to seek out additional resources to gain a reasonable understanding of influence functions. There are several missed opportunities for short exposition about influence functions, such as at line 85 or following Definition 2 at line 111. The writing style and organization of the paper conceal the machine learning advancements and how these advances differ from the previous work. For example, by placing the "Comparison with Other Approaches" at the near end of the paper, a reader may be mislead into thinking that the work presented in sections 2 through 4 are either the work of the authors or standard in this sub-field of influence functions. After reading section 5, I was aware of the differences between this work and previous work and thus, I was better prepared to re-read sections 2 through 4 than I was after reading those sections the first time. I recommend both moving section 5 to earlier in the paper (either as part of section 1 or immediately following section 1) and to add a bit of exposition at the beginning of section 3 "Estimating Functionals" that makes it clear where the authors contributions begin. There are many impressive facets of this work. The results on the ETH-80 dataset are impressive in context of the previous work presented in this paper. I wish there had been additional exposition about this experiment in the main paper. I also like that they present an open question (at line 267) that is a result of their work. This makes it clear that their work has pushed the boundary for the field and has opened further research avenues.


Approximating Sparse PCA from Incomplete Data
We study how well one can recover sparse principal components of a data matrix using a sketch formed from a few of its elements. We show that for a wide class of optimization problems, if the sketch is close (in the spectral norm) to the original data matrix, then one can recover a near optimal solution to the optimization problem by using the sketch. In particular, we use this approach to obtain sparse principal components and show that for m data points in n dimensions, O( −2k max{m, n}) elements gives an -additive approximation to the sparse PCA problem (k is the stable rank of the data matrix). We demonstrate our algorithms extensively on image, text, biological and financial data. The results show that not only are we able to recover the sparse PCAs from the incomplete data, but by using our sparse sketch, the running time drops by a factor of five or more.

Summary: This papers provides an algorithm to recover sparse eigenvectors of covariance matrix (those principal components that have sparse weights in the span of data points) usinga small ( or corrupted) subsample from the data. Thealgorithm is based on sampling elements from covariance matrix to build its "sketch" and performing sparse PCA on that sketch. The authors show that under reasonable stability assumptions and a specified sampling procedure: Greedy Sparse Sampling that sets small elements to zero and Hybrid (l1,l2) Sampling that selects larger elements with higher probability, sparse PCA on a "sketch" matrix approached sparse PCA on the original covariance matrix.The theoretical guarantees (for Hybrid (l1,l2) Sampling) are given in Theorem 4, which says that for a sufficiently large sample the sketch of covariance matrix is close to the original matrix on the order of \epsilon*||A||_2, where ||A||_2 is the 2-norm of original covariance matrix and \epsilon is the accuracy parameter of the algorithm.Quality & clarity: Overall this paper is of good quality, the theoretical analysis of the stability of "sketch" matrix relative to original covariance matrix in Theorems 2 and 4 is rigorous.The empirical results on four major datasets are clearly illustrated as well as the experimental setting is clear to the reader.Originality: I think this paper lacks a certain degree of theoretical originality, the concentration bounds that they show are quite straightforward to derive, for instance the concentration on the difference of true covariance and its approximation ||A-\tilde{A}||_{2}. Though from empirical point of view it is original since sparse PCA is inherently NP-hard problem and solving it through corrupted or small sample is an original approach.Significance: I believe the suggested algorithm is significant, especially for the real assets where instances have missing features. But not only for that - in any application where sparse PCA is required, but too long to solve due to NP-hardness,we can just use sub sampling suggested in this paper for a fast approximation of sparse PCA. However, there are still some flaky details: definition of sparse principal components is a heuristic in this paper, if we employ a different heuristic, do the results get worse? The authors report 5x improvement in the computational time, but that foes not mean much, it is more relevant to know ho does the runtime scale with input size? Also, for the real applications with corrupted data you do not get to sample from the original covariance matrix, the sample is already given to you - can stability be guaranteed for that case?


Randomized Block Krylov Methods for Stronger and Faster Approximate Singular Value Decomposition
Since being analyzed by Rokhlin, Szlam, and Tygert [1]  and popularized by Halko, Martinsson, and Tropp [2], randomized Simultaneous Power Iteration has become the method of choice for approximate singular value decomposition. It is more accurate than simpler sketching algorithms, yet still converges quickly for any matrix, independently of singular value gaps. AfterÕ(1/ ) iterations, it gives a low-rank approximation within (1 + ) of optimal for spectral norm error. We give the first provable runtime improvement on Simultaneous Iteration: a randomized block Krylov method, closely related to the classic Block Lanczos algorithm, gives the same guarantees in justÕ(1/ √ ) iterations and performs substantially better experimentally. Our analysis is the first of a Krylov subspace method that does not depend on singular value gaps, which are unreliable in practice. Furthermore, while it is a simple accuracy benchmark, even (1 + ) error for spectral norm low-rank approximation does not imply that an algorithm returns high quality principal components, a major issue for data applications. We address this problem for the first time by showing that both Block Krylov Iteration and Simultaneous Iteration give nearly optimal PCA for any matrix. This result further justifies their strength over non-iterative sketching methods.

The authors discuss SVD approximation using a (randomized) Block Lanczos algorithm. The idea for the algorithm is not new (has been proposed before) but the author's claim of proven convergence guarantee is better than the (state-of-the-art) guarantee for the simultaneous power iteration.The proofs are relegated to the full version of the paper, and while that would not prevent acceptance (in the minds of this reviewer), when the major contribution is in these proofs (with concerns regarding the experiment, applicability, see later) then this makes the paper less suitable for NIPS.One compelling insight is that to compute a low rank approximation (classically done via a partial SVD) one may not necessarily need to produce exact (and orthogonal etc.) singular vectors (and then form the low rank approximator) and therefore the (numerical, orthogonality) issues arising from close singular values may not prevent the finding of a low rank approximation itself efficiently (and for any matrix). However, this is not how the authors communicated their message, case in point: low rank approximation does not get mentioned in the title and only indirectly in the abstract when referring very briefly to PCA.It is well known that the Lanczos (block Lanczos) algorithm suffers from numerical issues manifesting itself in losing orthogonality (in floating point arithmetic) that is guaranteed in exact arithmetic. A comprehensive read on this topic is "Lanczos Algorithms for Large Symmetric Eigenvalue Computations" by J.K. Cullum and R.A Willoughby (SIAM classics in applied mathematics, 41). Various degrees of reorthogonalization have been considered to address this issue, which is of course expensive and limit the number of eigenvalues that can be computed to the required accuracy. Paige in 1980 have shown what exactly is behind the loss of accuracy and it is not simply the result of roundoff error but the interaction between roundoff and convergence of some eigenvalues.The authors of this paper do not discuss the problem with loss of orthogonality and simply state they reorthogonalize. This is not adequate in this reviewer's opinion. Furthermore, the complexity computation does not consider the reorthogonalization work and therefore can be labeled as misleading. Despite the above this reviewer thinks that the effort to apply block Lanczos is important and there may be a way to achieve the complexity claimed by the authors, meaning not doing full reorthogonalization.It could be valuable to see the author's experiments regarding the implementation when no reortogonalization is used. Did they run into the numerical issues prompting the reorthogonalization?Section 2.2 the authors discuss stronger guarantees for low rank approximation. This is an interesting and valuable section with the introduction of per vector guarantee and the discussion of failures of prior low-rank approximation guarantees in practice.The authors augment their theoretical work with a valuable experiment section, that shows the improved convergence. However, as mentioned above, using iteration count hides the costs of reorthogonalization and therefore more insightful discussion is warranted to discuss practical applicability.The paper is well organized and the presentation is good. There are several typos and inconsistencies, listed below, that should be addressed by the authors to improve the menuscript.38: Why mention rank r here at all? In addition k is used for rank later (see line 72). This can just cause slight confusion.39: The authors (at least consistently) use a weird symbol to denote matrix transpose. Superscript T should work fine instead.114: This reviewer expected K to have \Pi as well as shown on line 273.119: This reviewer is confused why take the top k singular vectors of the k x k matrix M?159: "instead of multiplicatively like" The reviewer understands that the per vector guarantee uses \sigma_{k+1} and not ]\sigma_i but what does multiplicatively has to do with this?343, 373: There is more discussion need regarding reorthogonalization and its impact on computational cost.424: This was published in 2010.429: The author's first name should be: Tam\'as.The line numbers below correspond to the full version:507: Instead of just telling about the existence of p_1, the authors could give some motivation so that the reader knows somewhat why are we doing this. While this becomes clear later, it is good to give motivation to help the reader not to be puzzled.524: It is quite unexpected and (not well defined in this reviewer's opinion) to use x < O(y). This reviewer understands that the authors try to convey that there is a constant when moving to the last inequality. Using big O in this manner is casual at best.1070: This reviewer finds the formulation and proof a bit cumbersome. The use of \eta is not really necessary is it? One could say instead:     ||A-B||_2^2 - ||A-A_k||_2^2 <= ||A-B||_F^2 - ||A-A_k||_F^2The proof could be made more streamlined if continuing the above thought. This reviewer also suggests to recall (as it was mentioned before) that A_k is the best rank -k approximation and that ||A-A_k||_2^2 = \sigma_{k+1}^2, and similar for the Frobenius norm.Additional comments following the author response and reviewer discussions.I revised my review based on the discussions and the author responses. I change my rating to a 5.However, I still think that with a major revision this could be a very valuable paper. My driving concern is that once the decision is made, there is no incentive to make the changes necessary. (At least that was my experience last year when reviewing.) So I am keeping my score as a slightly below acceptance. (If this were a journal publication, my call would recommend acceptance with a major revision.)I implemented the algorithms and I believe that it has merit and could be useful in actual applications.I went through the complexity computations. Please stay with me so I can make my point. Instead of using \epsilon I find it much more informative if we look at the cost in terms of q (number of iterations) and the cost of the matrix vector computation for A (and A') that I will denote by m. (m = 2*nnz(A) which I would expect to be about c*n assuming a constant number of non-zeros per column.)Alg 1, Simultaneous iteration has cost breakdown: m(2*q+1)k    - to form K 2nk^2        - compute QR to get Q A'Q          - km form M       - dk^2 SVD of M     - 2k^3 form Z       - 2nk^2Assuming k << q < n, the dominating term will be forming of K.Alg 2, Block Lanczos has cost breakdown: m(2*q+1)k    - to form K (but also note much more memory needed) 2nq^2k^2     - compute QR to get Q A'Q          - qkm form M       - dq^2k^2 top k SVD of M - 2q^2k^3 form Z       - 2nqk^2(Small note: forming M can be done by applying A' then A and then Q instead, the term is not dominating so we do not discuss which is better.)The dominating term here appears to be the QR computation assuming m is about c*n.The achieved accuracy is C_SI* log(d)/q and (C_BL*log(d)/q)^2, where C_SI, C_BL are constants for the algorithms respectively. This shows however that the accuracy achieved per computational effort is not satisfactory:     effort               accuracy (guarantee) SI cn(2q+1)k            C_SI*log(d)/q BL 2nq^2k^2             (C_BL*log(d)/q)^2Therefore it would seem that using SI with q^2 iterations is better. Of course the above ignores that in practice performance depends on use of back storage and in general memory etc. (But I am not the one submitting the paper.)This however tells me that the approach to use QR to orthogonalize K is flawed. After all, the insight of the Lanczos method is in computing an orthogonal base efficiently. Here we use QR and not a recurrence. I for one even find the name Block Lanzos for this process misleading. It is a Krylov space based method, but no Lanczos recurrence about it as proposed. (However, this bad terminology could have gotten hold in the literature earlier and it is not the authors fault.)I believe there is an efficient way of producing a Q that is "good enough" (in presence of numerical issues) that would work in practice, but more work is needed. I note that the idea has been out there before using the Krylov space, the trick is to make it work efficiently.Some more comments below that would further improve the paper (in addition to the previous ones).l. 115: Consider defining K with going up to q-1 only. As is written K is an n x (q+1)*k matrix. Otherwise, q downstream needs to be updated to (q+1) for correctness.l 378: Compare the algorithms on a computational effort versus error plot as well as empirical time versus error plot. (The latter requires a faithfully good implementation of both algorithms, but not state-of-the art, just fair to the point being made.)Emphasize that A is accessible as an operator through its action (matrix-vector product) in the targeted applications and that k is intended to be very small compared to n.***The implementation of the algorithms and the test script for reference:SI.m: function [Z] = SI(A, k, q)    [n, d] = size(A);P = randn(d, k);B = A*A'; C = A*P; K = B*C; %keyboard;[Q, R] = qr(K); Qk = Q(:, 1:k);M = Qk'*B*Qk;[U, S, V] = svd(M);Uk = U(:, 1:k);Z = Qk*Uk; return;**** BL.m: function [Z] = BL(A, k, q)    [n, d] = size(A);P = randn(d, k);B = A*A'; C = A*P; K = zeros(n, q*k); for i = 0:q-1   K(:, i*k + 1:(i+1)*k) = C;   C = B*C; end[Q, R] = qr(K); Qk = Q(:, 1:q*k);M = Qk'*B*Qk;[U, S, V] = svd(M);Uk = U(:, 1:k);Z = Qk*Uk; return;test.m: *** rand("state", 423);n = 1000; d = 300; k = 5; tol = 1e-3; q = log(d)/sqrt(tol)*0.01 q = 10;A = randn(n, d); [U, S, V] = svd(A); Uk = U(:, 1:k); Ak = Uk*Uk'*A;Z1 = SI(A, k, q); Z2 = BL(A, k, q);norm(A - Z1*Z1'*A)/norm(A - Ak) norm(A - Z1*Z1'*A, 'fro')/norm(A - Ak, 'fro')norm(A - Z2*Z2'*A)/norm(A - Ak) norm(A - Z2*Z2'*A, 'fro')/norm(A - Ak, 'fro')


Predtron: A Family of Online Algorithms for General Prediction Problems
Modern prediction problems arising in multilabel learning and learning to rank pose unique challenges to the classical theory of supervised learning. These problems have large prediction and label spaces of a combinatorial nature and involve sophisticated loss functions. We offer a general framework to derive mistake driven online algorithms and associated loss bounds. The key ingredients in our framework are a general loss function, a general vector space representation of predictions, and a notion of margin with respect to a general norm. Our general algorithm, Predtron, yields the perceptron algorithm and its variants when instantiated on classic problems such as binary classification, multiclass classification, ordinal regression, and multilabel classification. For multilabel ranking and subset ranking, we derive novel algorithms, notions of margins, and loss bounds. A simulation study confirms the behavior predicted by our bounds and demonstrates the flexibility of the design choices in our framework.

The paper proposes a new general perceptron-like algorithm, Predtron, for learning problems with large prediction and label spaces possibly of combinatorial nature, where predictions and labels are not necessary the same objects. First, a general mistake bound is given for the case in which the data is separable with a margin. Then, the algorithm is instantiated for several specific learning problems, such as binary classification (with or without a reject option), multiclass classification, ordinal regression, multilabel classification, subset ranking and multilabel ranking. The paper ends with a computational experiment.The paper is very nicely written, clear and easy to follow. The results look technically sound (although I did not check all the proof in the appendix). It is hard for me to assess the novelty of the proposed algorithm. The algorithm is a version of perceptron based on a generalization of hinge loss for binary classification. Such a generalization was already considered before (mainly in the context of structural SVMs), and is usually referred to as structural hinge loss. I think the contribution here is to define separate prediction and label spaces and explicitly use representation of predictions (which is usually hidden in the feature vector for SVMs). As far as I can tell, the mistake bound for the algorithm is a novel, original result. The proof is actually quite simple and more or less follow standard techniques from the analysis of online gradient descent (and its generalization, online mirror descent, for Predtron with a link function) and from the mistake-bound analysis of the original perceptron algorithm.What is a bit dissatisfying is that both the theoretical analysis and the experiment only concern a very special case of linear separability with margin. This is a very strong assumption on the data, and rarely appears in practice. For instance, the labels (relevance judgments) generated in the experiment are deterministic functions of the input X, so there is no label noise involved. It would be nice to see a counterpart of Theorem 1 in the nonseparable case, where the regret would be expressed as the total loss of the algorithm minus the total phi-loss (structural hinge loss) of the best W. A bound on such defined regret can be obtained for the standard perceptron algorithm (see e.g. Cesa-Bianchi, Lugosi: "Prediction Learning and Games", Chapter 12).The results on subset ranking are clearly interesting, especially the analysis of representation function and the value of the margin (Lemma 4). However, from what I know, solving max within a surrogate for NDCG through a linear assignment problem is known; see, e.g.: Chapelle, Le, Smola: Large margin optimization of ranking measures Le, Smola, Chapelle, Teo: Optimization of Ranking Measures


Latent Bayesian melding for integrating individual and population models
In many statistical problems, a more coarse-grained model may be suitable for population-level behaviour, whereas a more detailed model is appropriate for accurate modelling of individual behaviour. This raises the question of how to integrate both types of models. Methods such as posterior regularization follow the idea of generalized moment matching, in that they allow matching expectations between two models, but sometimes both models are most conveniently expressed as latent variable models. We propose latent Bayesian melding, which is motivated by averaging the distributions over populations statistics of both the individual-level and the population-level models under a logarithmic opinion pool framework. In a case study on electricity disaggregation, which is a type of singlechannel blind source separation problem, we show that latent Bayesian melding leads to significantly more accurate predictions than an approach based solely on generalized moment matching.

This paper proposes a method to integrate two prior belief under different views and applies it to a specific engineering problem called energy disaggregation. The method is based on an approach called Bayesian melding proposed previously. The authors replace a fixed prior in Bayesian melding with a prior having latent variables, and MAP-estimate the latent variables. The result shows the proposed method may improve the separation of energy consumption timeseries of individual appliances from total energy.The energy disaggregation problem is interesting in its own right and it is understandable that the integration of different types of prior belief should be important for this kind of problem. However, the proposed method is seen just an application of Bayesian melding once the latent variables in the prior have been fixed, so the method is quite trivial from general point of view. On the other hand, the method might contribute to the specific field where the energy disaggregation problem is relevant, while it was unclear to me from the manuscript.Although the paper is well written overall, the presentation of the background and the explanation of Bayesian melding (Sections 1 and 2) should be improved. The introduction is not very specific so it was difficult at the first reading to see what those methods mentioned here (posterior regularization, learning with measurements, etc. and also Bayesian melding) will actually do. Section 2 is quite problematic as it only gives the formula of Bayesian melding. The authors should introduce when and why having two different priors should be reasonable, and how it is related to the notion of individual and population mentioned in Introduction. Some more justification about the form of \tilde{p}_S(S) should also be necessary.Minor comments: - Sec.2: usuall -> usually, invertable -> invertible - Give the definition of c_\alpha - Is there any reasonable method to select alpha?


No-Regret Learning in Bayesian Games
Recent price-of-anarchy analyses of games of complete information suggest that coarse correlated equilibria, which characterize outcomes resulting from no-regret learning dynamics, have near-optimal welfare. This work provides two main technical results that lift this conclusion to games of incomplete information, a.k.a., Bayesian games. First, near-optimal welfare in Bayesian games follows directly from the smoothness-based proof of near-optimal welfare in the same game when the private information is public. Second, no-regret learning dynamics converge to Bayesian coarse correlated equilibrium in these incomplete information games. These results are enabled by interpretation of a Bayesian game as a stochastic game of complete information.

In game theory, one can show that under certain smoothness assumptions, the so called price of anarchy of a full information game is bounded. The price of anarchy is the ratio of the optimal social welfare over the worst social welfare achievable under certain type of equilibrium. The literature on this subject is quite extensive and bounds on the price of anarchy have been shown for the large class of coarse correlated equilibria. In the world of games of incomplete information, however, these results do not exist and only bounds on the price of anarchy of Bayes-Nash equilibria exist. The main contribution of this paper is to provide a bound on the price of anarchy for coarse correlated equilibria. The way this is done is by reducing these game to several full information games.The paper is really hard to read and seems to lack generality. Even the setup was not completely clear to me. What do the authors mean by"F_i be the empirical distributions of values in population P_i" which samples are being used to talk about the empirical distribution? I think that the main issue of the paper comes from Lemma 10. The hypothesis of the lemma are not completely clear to me. First of all, they say that D^T is an empirical distribution of plays and they say if a subsequence of D^T converges in distribution to D then ... However, D^T is a random measure so is the convergence in distribution a.s? Must the subsequence be the same for all values of D^T? This same type of questions arise on equation (10) after taking the limit. In a way they are taking the limit of the formE_{s \sim D^T}[f_T(s)] and they claim that because f_T \right arrow f a.s. and D^T \rightarrow D in distribution (which again does not make much sense), then the above term will converge to E_{s \sim D} [f(s)]. It is not entirely clear to me that this will hold in general and if it does then the authors should provide a better explanation.The rest of the paper seems to be correct so if the authors can address my comments on Lemma 10 I would consider accepting the paper. Nevertheless, I don't believe that the subject of the paper would be of much interest to the NIPS community.


On Variance Reduction in Stochastic Gradient Descent and its Asynchronous Variants
We study optimization algorithms based on variance reduction for stochastic gradient descent (SGD). Remarkable recent progress has been made in this direction through development of algorithms like SAG, SVRG, SAGA. These algorithms have been shown to outperform SGD, both theoretically and empirically. However, asynchronous versions of these algorithms-a crucial requirement for modern large-scale applications-have not been studied. We bridge this gap by presenting a unifying framework for many variance reduction techniques. Subsequently, we propose an asynchronous algorithm grounded in our framework, and prove its fast convergence. An important consequence of our general approach is that it yields asynchronous versions of variance reduction algorithms such as SVRG and SAGA as a byproduct. Our method achieves near linear speedup in sparse settings common to machine learning. We demonstrate the empirical performance of our method through a concrete realization of asynchronous SVRG.

"Light" review:This work proposes a framework for linearly-convergent stochastic gradient methods that allows asynchronous computation. The paper analyzes a generalization of SVRG and SAGA, showing what speedups can be obtained.I like the statement of the general framework, and how it includes many of the previous algorithms. However, I think if you made two small modifications you could include basically all of the linearly-convergent primal methods:1. The framework doesn't seem to include SAG because the update is a bit different. In particular, SAG weighs the two $\nabla f_{i_t}$ terms by (1/n). If you added a weight to Algorithm 1 on these terms, it would allow you to include SAG. Further, adding these weight would allow you to include non-uniform sampling in the framework as in [27].2. In the update x^{t+1} = x^t - \eta(...), you could replace x^t by a weighted average of x^t and the alpha_i^t. If you do this, you could include MISO in the framework. MISO would correspond to weights of 1/n on the alpha_i^t, and all other methods place all their weight on x^t.I did not check the details, but all the results seems reasonable and plausible.The experiments are not terribly convincing in terms of utility of the method. On most datasets there only seems to be a small improvement over the locked SVRG (less than a factor of 2 in 3/4 cases). It would be interesting to see the difference if a larger number of threads were available.


Bandit Smooth Convex Optimization: Improving the Bias-Variance Tradeoff
Bandit convex optimization is one of the fundamental problems in the field of online learning. The best algorithm for the general bandit convex optimization problem guarantees a regret of e O(T 5/6

As I have stated in review summary, the significance of the paper, as least in my opinion, cannot be questioned. It is a simple question of whether the proofs are correct. I will highlight a couple of points which I believe are mistakes. Other than that, I have a few minor questions regarding presentation of the paper (which are not important in terms of accept/reject decision, but if the paper gets in, will be important in terms of clarity).Line 322-323- The authors state that by definitions of \gar{g}_t and the \alpha variables, it is true that \sum_{s=1}^t \alpha_{s,t} \hat{g}_s= \sum_{s=1}^t \bar{g}_s. In fact, this is an important claim because then the dual averaging step reduces to an operation with gradient estimates \bar{g}_1,...\bar{g}_T.I simply cannot see how this is true. I feel this is basic algebra. First of all, even for the equality to hold, we must have \eta \sum_{s=1}^t \bar{g}_s (I can understand missing \eta is a typo). However, even with that, I dont think the equality holds. From definition, it can be seen that \eta \sum_{s=1}^t \bar{g}_s=\sum_{s=1}^t \eta \frac{t-s+1}{K+1} \hat{g}_s. Thus, for each \hat{g}_s, the coefficient associated with it is :\eta \frac{t-s+1}{K+1}. However, for all s, the coefficient is not equal to \alpha_{s,t}. By definition, \alpha_{s,t}=\eta \frac{t-s+1}{K+1} only for s \> t-k. So, how is the equality holding when s\le t-k?Lemma 11: (8c)\le 0. When i looked at the proof of this statement, I once again felt there was a mistake. It is stated in the proof that sum of the \bar{f}_t(x^*) terms is equal to sum of the \hat{f}_t(x^*) terms. I dont see how that is true. This can be simply tested by taking T=2 and K=1 (allowed by condition of Thm. 9) By definitions, for T=2 and K=1, \sum_{t=1}^T \bar{f}_t(x^*)= \hat{f}_1(x^*) + (1/2) \hat{f}_2(x^*), which is obviously not equal to \hat{f}_1(x^*) + \hat{f}_2(x^*).I did not even look through the rest of the proofs. I find the two mistakes very surprising since they seem to be basic algebra. It is of course possible that the authors made some typos which have completely changed some important definitions. Otherwise, I will be happy to be corrected.In terms of presentations, I have a few small questions: 1. In abstract, in line 026, it should be that the regret guarantee is for smooth convex functions, not all convex functions. For the general Lipschitz BCO problem, the lower bound might still be \Omega (T^{2/3}) (or even higher, since the best known result is Flaxman's O(T^{3/4}). 2. I am not sure what Thm.5 means (or rather, the definition of K_{y,\epsilon}. In the definition of K_{y,\epsilon}, there is an \alpha term present and no epsilon term. Is \alpha= \epsilon? It seems to be a typo. 3. Thm 9. In the regret bound, the term \epsilon appears. Nowhere in the algorithm or in the statement of Thm.9 do we have \epsilon. I understand that \epsilon probably appears while using an upper bound on the self-concordant R(), but this is quite poor in terms of presentation. Please modify the statement of Thm 9 to explicitly mention \epsilon. Or dont mention \epsilon at all and use T (like in Saha and Tewari).4. In Saha and Tewari's paper, the authors dont analyze regret directly in terms of x^*. They instead consider a term \bar{x}, which is 1/\sqrt(T) away from boundary of K. This allows R(\bar{x}) \le 2 \new log(T). How do you use x^* directly?5. Please provide definition of Minkowski functional in supplement to make the paper self-containing. Specifically, I would like to know what is the Minkowski's func. associated with a Dikin's ellipsoid. Like specifically what does the functional look like for Dikin's ellipsoid?Update: It appears that the proofs are correct. I accept the paper.


Particle Gibbs for Infinite Hidden Markov Models
Infinite Hidden Markov Models (iHMM's) are an attractive, nonparametric generalization of the classical Hidden Markov Model which can automatically infer the number of hidden states in the system. However, due to the infinite-dimensional nature of the transition dynamics, performing inference in the iHMM is difficult. In this paper, we present an infinite-state Particle Gibbs (PG) algorithm to resample state trajectories for the iHMM. The proposed algorithm uses an efficient proposal optimized for iHMMs and leverages ancestor sampling to improve the mixing of the standard PG algorithm. Our algorithm demonstrates significant convergence improvements on synthetic and real world data sets.

In this paper the authors makes use of the Particle Gibbs with ancestor sampling algorithm to solve the inference problem in the iHMM. The novelty lies in the application of Particle Gibbs with ancestor sampling via the standard stick-breaking construction for the HDP. The authors also exploits the fact that the model is simple enough to allow for better (i.e. better than the bootstrap) proposal distributions to be used, which is good to see.The paper is well written a pleasant read. As always the degree of novelty can be discussed. In this paper the authors takes an existing inference algorithm (PG with ancestor sampling) and applies it to an existing model (iHMM) in a straightforward manner. This particular combination has not appeared before (as far as I know), so in that sense it is novel, but the ``degree of novelty'' is just that, i.e. applying algorithm X to model Y.As already mentioned above it is good to see that the authors are exploiting the analytical tractability inherent in the model to construct a better proposal distribution. This should always be done when it is possible. If you want more background on this you can consult the literature on the so called ``fully adapted'' particle filter. I think you might be able to adapt also the resampling step for your model. Anyhow, just a thought.A detail: The authors writes (already in the abstract) that ``... leverages ancestor sampling to suppress degeneracy of the standard PG algorithm.'' Technically this is not correct. Note that the PG sampler with ancestor sampling also degenerates, but importantly it (with a very high probability) degenerates to a new trajectory in each iteration. The fact that the algorithm degenerates to a different trajectory in each iteration is indeed what improves the mixing when compared to the original PG sampler.


On the Optimality of Classifier Chain for Multi-label Classification
To capture the interdependencies between labels in multi-label classification problems, classifier chain (CC) tries to take the multiple labels of each instance into account under a deterministic high-order Markov Chain model. Since its performance is sensitive to the choice of label order, the key issue is how to determine the optimal label order for CC. In this work, we first generalize the CC model over a random label order. Then, we present a theoretical analysis of the generalization error for the proposed generalized model. Based on our results, we propose a dynamic programming based classifier chain (CC-DP) algorithm to search the globally optimal label order for CC and a greedy classifier chain (CC-Greedy) algorithm to find a locally optimal CC. Comprehensive experiments on a number of real-world multi-label data sets from various domains demonstrate that our proposed CC-DP algorithm outperforms state-of-the-art approaches and the CCGreedy algorithm achieves comparable prediction performance with CC-DP.

The paper deals with the construction of optimal classifier chains for multi-label classification. The first part of the paper provides a bound on the error of the classifer chain in terms of the margins over the labels. This result is used in the second part to derive a DP and greedy algorithm. The experimental section shows that that the DP approach has the best results in 5 out of 8 datasets and the greedy approach in 1.Quality: The bound on error involves interesting ideas. Hence the paper scores high on quality.Clarity: The exposition is clear. The authors should address the following questions.1. The greedy approach performs better than DP on 2 out of 8 datasets. Some explanation will be helpful. 2. Comparison of actual runtimes of greedy vs DP will help.The paper has some typos. For example, line 313 should read "\gamma^1_i represents ..."Orginality: HighSignificance: The techniques used in the paper have theoretical significance. I have not seen many people using multilabel classification in practice.


Matrix Completion with Noisy Side Information
We study the matrix completion problem with side information. Side information has been considered in several matrix completion applications, and has been empirically shown to be useful in many cases. Recently, researchers studied the effect of side information for matrix completion from a theoretical viewpoint, showing that sample complexity can be significantly reduced given completely clean features. However, since in reality most given features are noisy or only weakly informative, the development of a model to handle a general feature set, and investigation of how much noisy features can help matrix recovery, remains an important issue. In this paper, we propose a novel model that balances between features and observations simultaneously in order to leverage feature information yet be robust to feature noise. Moreover, we study the effect of general features in theory and show that by using our model, the sample complexity can be lower than matrix completion as long as features are sufficiently informative. This result provides a theoretical insight into the usefulness of general side information. Finally, we consider synthetic data and two applications -relationship prediction and semisupervised clustering -and show that our model outperforms other methods for matrix completion that use features both in theory and practice.

This paper is suitable for all level of readers.Overall This paper is very well written. The idea is novel while some assumptions may not be quite valid, For example, the authors assume the M matrix to be low rank, and did not prove that.I understand R matrix is low rank, but why the smaller matrix M is also low rank?I would expect more elaboration on this assumption.  The contribution and proposal is straight froward and the presentation is easy to follow.I like the idea of updating the side information, and setting of the problem is nicely formulated.The comments to help further improvement is the author did not discuss the details about the implementation.Solving the optimization problem is not trivial, especially get to the global optimal.I do believe there are some constraints or further assumptions needed before this problem is solvable.The variation of solutions might be another problem.how to find the global optimal.the idea is quite original, especially the estimation of the transform matrix.While the authors may need to read some of the related work on noisy side information.Some may be quite inspiring as well in how to solve this type of problem.And it is highly recommend the authors compare and discuss similar methods in literature and experimental section. For example: Matrix Completion with Noise Noisy Matrix Completion Using Alternating Minimization Distance Metric Learning from Uncertain Side Information with Application to Automated Photo Tagging\the problem itself is common and practically useful. the idea seems interesting but involves some assumption, which is my major concern can be some problem affecting the results.The authors needs further investigate, either prove the assumptions are valid, or remove the assumptions and reformulate the problem in a more natural / organic way.


Finite-Time Analysis of Projected Langevin Monte Carlo
We analyze the projected Langevin Monte Carlo (LMC) algorithm, a close cousin of projected Stochastic Gradient Descent (SGD). We show that LMC allows to sample in polynomial time from a posterior distribution restricted to a convex body and with concave log-likelihood. This gives the first Markov chain to sample from a log-concave distribution with a first-order oracle, as the existing chains with provable guarantees (lattice walk, ball walk and hit-and-run) require a zerothorder oracle. Our proof uses elementary concepts from stochastic calculus which could be useful more generally to understand SGD and its variants.

The authors consider the projected Langevin Monte-Carlo (PLMC) algorithm for sampling from a log-concave density with a compact support. They establish a polynomial-time bound for sampling with a prescribed accuracy. The result is new, nontrivial and, in my opinion provides a very interesting contribution to the theory of computational statistics/machine learning. I went through the proof (without reading the parts which are in the supplementary material) and checked all the details. I have found only a couple of small typos (cf. below). Not only the proof is correct, but also it contains a very elegant argument that makes use of the Skorohod embedding. I am sure any mathematically driven person will appreciate this argument.Here are some minor remarks: - I recommend to add somewhere in the discussion that the PLMC needs not only a first-order oracle but also a projector to the compact set under consideration.- Concerning the discussion on H&R in Section 1.2: the authors seem to claim that H&R requires a zeroth-order oracle, but my understanding is that H&M requires to sample from the restriction of the distribution on the segments. Could the authors comment on the relationship between these two tasks? - Beginning of section 2.1: why do you need to introduce the pair (x,phi)? It is my impression that it suffices to introduce x. Then you can remove the equation of line 184 and replace phi by x-w in the equation on line 187.- line 210: "Dirac" instead of "dirac" -line 225: "paths" instead of "path" - line 244 and Eq (3): in these two formulae there is a \bar W_T that should be replaced by \bar W_t (small t instead of capital T). - line 298: please provide a precise reference for the version of the Khintchine inequality you use.- line 321: could you provide a reference where it is clearly stated that the uniform distribution on a compact set is a stationary distribution for the reflected BM ?N.B. In the confidence evaluation above I have put 4 (and not 5) just because I cannot be absolutely certain of my evaluation of the paper without carefully reading all the details of the proofs which are put in the supplementary material.