Competitive Distribution Estimation: Why is Good-Turing Good
Estimating distributions over large alphabets is a fundamental machine-learning tenet. Yet no method is known to estimate all distributions well. For example, add-constant estimators are nearly min-max optimal but often perform poorly in practice, and practical estimators such as absolute discounting, Jelinek-Mercer, and Good-Turing are not known to be near optimal for essentially any distribution.

The paper gives justification for the widespread use of the Good-Turing estimator for discrete distribution estimation through minimax regret analysis with two comparator classes. The paper obtains competitive regret bounds that lead to a more accurate characterization of the performance of the the Good-Turing estimators and in some cases is much better than the best known risk bounds. The comparator classes considered are estimators with knowledge of the distribution up to permutation, and estimators with full knowledge of the distribution, but with the constraint that the must assign the same probability mass to symbols appearing with the same frequencies. While I like the exposition in the paper, I think there are several nice results in the appendix that should be highlighted that also clear up some confusions in the paper. For example, the fact that the optimal natural estimator can be explicitly computed and its risk can be bounded is very useful. First, it means that we actually obtain an "instance-specific" risk bound for the Good-Turing estimator by combining Lemma 10 with Theorem 2. It would be nice to give examples of this bound being much tighter than the best log(k/n) bound on the minimax risk.Major comments: 1. In Figure 1, what is the estimator designed with prior knowledge of the partition? Is the red line a theoretical curve or an empirical one? Is there any analysis for the case where the partition is known? Roughly, my question is whether that red curve actually lower bounds the minimax risk here. I know you are using the lower bound r_n(\Delta_k) \ge r_n(P), but I am not sure how you are calculating the min_q in the definition of r_n(P). It is not sufficient to use an upper bound (i.e. a sub-optimal estimator) if the claim is that the red curve lower bounds the r_n(\Delta_k).I think this is answered in the proof of Lemma 4 in the appendix but it should be made clear in the main paper.2. In the experiments it would be helpful to make it clear what things are known to the estimators and what aren't. I believe the quantity S(t) is not known to fully data-driven estimators and only to the comparator in the natural class. Is that correct? 3. More generally, I noticed that some notation is defined only in the appendix and not in the main paper and vice versa. It would be nicer and aid readability if all of the notation was in the main paper (and maybe even remind the reader from time to time).Minor comments: 1. In the abstract you use k and n but those variables have not been defined yet.2. "The distributions resulting in r_n(\Delta_k)." I think you mean to say the distributions achieving the max in the definition. 3. The claim that "which in turn is better than the performance of any data-driven estimator designed without prior information" is technically not true for the second comparator since in theory there could be an estimator which does not assign the same estimate to symbols with the same frequency. I do believe the prior information makes the problem easier but technically there could be a better estimator that does not obey this constraint. 4. Some typos in the first paragraph of section 2.3. Sentence beginning with "for example..." is not technically a sentence. No space after period. 5. Typo line 227. 6. n(x) undefined in the main text. I think it is the number of times symbol x appeared in the sample.


NEXT: A System for Real-World Development, Evaluation, and Application of Active Learning
Active learning methods automatically adapt data collection by selecting the most informative samples in order to accelerate machine learning. Because of this, real-world testing and comparing active learning algorithms requires collecting new datasets (adaptively), rather than simply applying algorithms to benchmark datasets, as is the norm in (passive) machine learning research. To facilitate the development, testing and deployment of active learning for real applications, we have built an open-source software system for large-scale active learning research and experimentation. The system, called NEXT, provides a unique platform for real-world, reproducible active learning research. This paper details the challenges of building the system and demonstrates its capabilities with several experiments. The results show how experimentation can help expose strengths and weaknesses of active learning algorithms, in sometimes unexpected and enlightening ways.

This paper describes an open-source platform, NEXT, for deploying general active learning algorithms in a real-world setting, where users or clients provide live, real-time feedback through a web interface to queries formed by active learning algorithms. Aiming at facilitating both research in active learning itself and research in other fields that use active learning, the NEXT system hides away engineering details such as load-balancing, logging, communication, locking, etc and lets researchers focus on algorithm or experiment design. Two use cases are presented to illustrate the capability of the system in comparing different algorithms in a real active learning setting, as opposed to many existing studies that conduct comparisons in a simulation mode.The system described in the paper is a great effort in facilitating active learning in the real world. Two suggestions on the presentation / doc side are as follows.(1) While the two example applications in the paper are nice, it would be great ifthere is also a template for a perhaps more typical active learning task, e.g., image or web page classification, that ML researchers/users can easily modify or extend.(2) The documentation of NEXT seems to focus on launching the system in the cloud, but most users would probably want to start by running the system locally to get a feeling of how it works. It is not clear from the documentation how to do that.


Gradient Estimation Using Stochastic Computation Graphs
In a variety of problems originating in supervised, unsupervised, and reinforcement learning, the loss function is defined by an expectation over a collection of random variables, which might be part of a probabilistic model or the external world. Estimating the gradient of this loss function, using samples, lies at the core of gradient-based learning algorithms for these problems. We introduce the formalism of stochastic computation graphs-directed acyclic graphs that include both deterministic functions and conditional probability distributions-and describe how to easily and automatically derive an unbiased estimator of the loss function's gradient. The resulting algorithm for computing the gradient estimator is a simple modification of the standard backpropagation algorithm. The generic scheme we propose unifies estimators derived in variety of prior work, along with variance-reduction techniques therein. It could assist researchers in developing intricate models involving a combination of stochastic and deterministic operations, enabling, for example, attention, memory, and control actions.

The manuscript proposes a formalism for computing stochastic estimates of gradients for loss functions.The formalism, referred to as stochastic computation graphs, is very general, applying to models with deterministic and stochastic components, and allowing the computation of gradient estimates for a broad range of models.Methods for deep networks, variational inference, and reinforcement learning are identified as special cases of the proposed framework.The proposed stochastic computation graphs are essentially Bayesian networks which may also contain deterministic nodes, and which are interpreted as encoding distributions over loss functions that are to be minimized.The gradient estimation algorithm extends backpropagation to the partially stochastic case covered by these models, by simply composing score function estimators and pathwise derivative estimators.The main result of the paper is a theorem stating that the gradient estimation algorithm recovers an unbiased estimate of the gradient for these models, under broad conditions.The formalism and algorithm are both simple and represent fairly small adaptions of existing ideas.However, the ideas are likely to have substantial practical impact by facilitating automatic optimization of a wide range of custom models and architectures.The approach is both elegant and general.The biggest weakness of the paper is a complete lack of empirical analysis.This cannot be excused by considering it a "theory paper," since the proposed algorithm is practically motivated, and derives much of its potential impact from its potential use.It would GREATLY strengthen the paper to include experimental results demonstrating that the method is competitive with state of the art special-purpose methods on a variety of real world problems.It would also be insightful to explore and graph its performance on randomly generated problems, varying difficulty parameters such as the depth of the networks, percent of stochastic nodes, variance of stochastic nodes, number of parameters, etc.The only mitigating factor regarding the lack of empirical results is that special cases of the method are identified in the literature, thereby indirectly demonstrating that it can have practical utility.Together with the potential impact of the work, this leads me to consider it for publication, despite the above reservations.


Revenue Optimization against Strategic Buyers
We present a revenue optimization algorithm for posted-price auctions when facing a buyer with random valuations who seeks to optimize his -discounted surplus. In order to analyze this problem we introduce the notion of ✏-strategic buyer, a more natural notion of strategic behavior than what has been considered in the past. We improve upon the previous state-of-the-art and achieve an optimal regret bound in O(log T + 1/ log(1/ )) when the seller selects prices from a finite set and provide a regret bound in e O( p T + T 1/4 / log(1/ )) when the prices offered are selected out of the interval [0, 1].

This paper considers the problem of a seller selling an item to a single strategic buyer over a period of T rounds. As in a classic bandit problem, the seller attempts to learn the optimal posted price for the item by running an explore/exploit problem over the set of possible prices. However the buyer, knowing that the seller is learning about his value distribution, may lie in the hopes that the seller will eventually settle on lower prices. This problem has seen some interest recently but the models tend to differ slightly. The three defining features of the model in this paper are:1. Buyers realized a value from their value distribution at each time step t. (as opposed to having a fixed value over all rounds).2. They discount their utility over time (this is actually common to all models, and a necessary component to any low-regret scenario).3. The buyer acts epsilon-strategically, meaning that if the truthful strategy (telling the truth for the remaining time period) guarantees a payoff within epsilon of the optimal strategy at any point, the buyer will use the truthful strategy. Under this model, the authors show that a modification to the the standard UCB algorithm, denote by UCB_L where L is a free parameter to be set later, and show that it achieves the best known regret within this class of models (although similar work has a slightly different model, so the comparison is not direct). Furthermore it achieves the optimal regret when the buyer acts truthfully. This is a nice paper that contributes to an interesting and compelling problem within the intersection of machine learning and game theory. The logic of the paper's main proof, which is a clever adaptation of the standard UCB proof, is well-written and easy to follow. The main result is not ground-breaking but it is a solid contribution to the study of online revenue optimization against strategic buyers. The result is perhaps a little incremental, in the sense that the main result is similar to other work but with a twist to the model, but I think the study of this problem is nascent enough to warrant such a contribution. The one comment I have is that the author's definition of epsilon-strategic buyers could be more explicit. I admit that I first only skimmed their definition because I assumed it was a simple adaptation of the notion of epsilon-best response (from a standard game theory model) but it's not. I could imagine other readers who are familiar with concepts like epsilon Nash equilibria also making the same mistake. I think the definition used in this paper is a fine one but it just might need to be spelled out a bit more. A few remarks:- The statement of lemma 1 reads "... for delta >0, the strategic regret of a buyer can be bounded...". I assume this should be the strategic regret of seller. If I misunderstood and it in fact should be the strategic regret of the buyer, please let me know. - On page 7, the set of inequalities that begins on like ~334: In the second inequality, the indicator variable reads "1_(v_i > p_2)". I assume the 2 in p_2 is a typo and should be just p.


High Dimensional EM Algorithm: Statistical Optimization and Asymptotic Normality
We provide a general theory of the expectation-maximization (EM) algorithm for inferring high dimensional latent variable models. In particular, we make two contributions: (i) For parameter estimation, we propose a novel high dimensional EM algorithm which naturally incorporates sparsity structure into parameter estimation. With an appropriate initialization, this algorithm converges at a geometric rate and attains an estimator with the (near-)optimal statistical rate of convergence. (ii) Based on the obtained estimator, we propose a new inferential procedure for testing hypotheses for low dimensional components of high dimensional parameters. For a broad family of statistical models, our framework establishes the first computationally feasible approach for optimal estimation and asymptotic inference in high dimensions.

A sparse EM procedure is proposed which modifies classical EM by truncating the parameter vector at each step to a desired sparsity level. Also proposed is an asymptotically normal statistic based on a decorrelated score for testing whether a specific entry of the parameter vector is non-zero. The guarantee provided for the sparse EM states that the estimate converges geometrically to the true value (in the number of iterations), up to a statistical error term which is like the square root of the sparsity level and the infinity norm of the statistical error in the M-step, which is typically only logarithmic in the ambient dimension.This follows on the heels of recent similar results for EM in the low-dimensional setting. As in the previous work, it is assumed that the Q function is sufficiently smooth and nearly quadratic in a certain basin of attraction, and that the algorithm is initialized in the same basin. The new condition is the infinity norm control on the error in estimating M. This condition is required only for sparse vectors, but for many models estimating M at all possible vectors is just as easy as at sparse vectors (this holds for the Gaussian mixture example considered, and for the non-gradient version of the mixture of regression model). This raises a question -- is the truncated EM algorithm necessary? Could a similar guarantee be obtained, for instance, by performing ordinary EM under the same conditions, followed by a single truncation step? EDIT: the authors' observation that linear regression is a special case here seems to be a convincing argument against a single truncation step.There may be some related work in sparse optimization that perhaps should be discussed (like "Sparse online learning via truncated gradient" Langford et al 2009, just as an example).The experimental section should state what initialization was used.


On the Accuracy of Self-Normalized Log-Linear Models
Calculation of the log-normalizer is a major computational obstacle in applications of log-linear models with large output spaces. The problem of fast normalizer computation has therefore attracted significant attention in the theoretical and applied machine learning literature. In this paper, we analyze a recently proposed technique known as "self-normalization", which introduces a regularization term in training to penalize log normalizers for deviating from zero. This makes it possible to use unnormalized model scores as approximate probabilities. Empirical evidence suggests that self-normalization is extremely effective, but a theoretical understanding of why it should work, and how generally it can be applied, is largely lacking. We prove upper bounds on the loss in accuracy due to self-normalization, describe classes of input distributions that self-normalize easily, and construct explicit examples of high-variance input distributions. Our theoretical results make predictions about the difficulty of fitting self-normalized models to several classes of distributions, and we conclude with empirical validation of these predictions.

This paper provides a theoretical analysis of self-normalized log-linear models, which are log-linear models trained with a penalty to keep the partition function close to one, so that scores can be directly interpreted as probabilities. These models are being used in the NLP community when the output space is large. The authors provide 1) a simple geometric characterization of exact and approximate self-normalization 2) a natural framework to characterize the tradeoff between fitting the data and keeping the log-partition function close to zero 3) some results showing that distributions that are either close to uniform or close to degenerate can be self-normalized (without losing too much in terms of likelihood). The paper is well written, it provides intuitive explanations for the results, examples and figures. I would have liked to see more discussion for the results at the end of section 4 (looks like it was finished in a hurry). As far as I know, the results are novel. The theoretical results shed some light on the role of self-normalization. The framework developed for the analysis is of (independent) interest and could be used by other researchers, e.g. to obtain deeper characterizations of the tradeoffs involved. This work could be of interest to people working on models where dealing with the intractable log-normalizer is the key computational bottleneck (on energy based models, noise constrastive estimation, etc). One limitation is that the results presented don't directly lead to new algorithms, and it's not clear to me if they are useful to inform the work of a practitioner (e.g, how to set parameters or design models). I am also not sure about the significance and the possible implications of Theorem 4. How does the model fit (data likelihood) enter the picture? Can you say something about small perturbations of \eta_0?More details should be given about the experimental evaluations. Currently, there is not enough information to replicate the results (what is the X, Y, features used etc.)in proposition 1, should it be ||\eta||_2 <= B ?


Subset Selection by Pareto Optimization
Selecting the optimal subset from a large set of variables is a fundamental problem in various learning tasks such as feature selection, sparse regression, dictionary learning, etc. In this paper, we propose the POSS approach which employs evolutionary Pareto optimization to find a small-sized subset with good performance. We prove that for sparse regression, POSS is able to achieve the best-so-far theoretically guaranteed approximation performance efficiently. Particularly, for the Exponential Decay subclass, POSS is proven to achieve an optimal solution. Empirical study verifies the theoretical results, and exhibits the superior performance of POSS to greedy and convex relaxation methods.

The paper is concerned with a new algorithm (named POSS) for small-sized subset selection based on Pareto optimization. The proposed algorithm is extensively analyzed, compared and tested. A detailed comparison is presented between POSS and the FR algorithm (Forward Regression) for which a theoretical approximation bound is known. The authors also give such a theoretical approximation bound for their algorithm and conclude that POSS does at least as well as the FR algorithm on any sparse regression problem. A second result is given on the exponential decay subclass (a subclass of sparse regression: the observation variables can be ordered in a line such that their covariances are decreasing exponentially with the distance): On this subclass POSS has strictly better guarantees than FR. The authors also show that POSS finds (again on the exponential decay subclass) an optimal solution. This guarantee cannot be given for the FR algorithm as demonstrated be the authors. This concludes the theoretical part which is followed by an empirical study: 4 different algorithms (POSS, FR, OMP, FoBa) and the (exact) optimal solution (OPT), where it could be calculated in acceptable time, are compared on 12 different data sets. POSS outperforms all 4 algorithms when calculating the squared multiple correlation (R2) which is equivalent to the mean squared error. Although the authors claim a significantly better performance in all 4 cases, the reviewer is still not fully convinced about the attribute 'significantly'. The empirical study is extensive: calculating R2 on test and training data, performance vs running-time analysis and discussion of a possible pitfall (sparsity alone may not be a good sparsity measure and thus better optimization does not always lead to better generalization performance) and a solution to that pitfall (introduce l2 norm regularization into the objective function).In general, the paper is written in a clear and transparent way, and it is easy to follow the main arguments. Minor Comment: Line 126 - shouldn't it spell 'subset' instead of 'subsets'?


Learning spatiotemporal trajectories from manifold-valued longitudinal data
We propose a Bayesian mixed-effects model to learn typical scenarios of changes from longitudinal manifold-valued data, namely repeated measurements of the same objects or individuals at several points in time. The model allows to estimate a group-average trajectory in the space of measurements. Random variations of this trajectory result from spatiotemporal transformations, which allow changes in the direction of the trajectory and in the pace at which trajectories are followed. The use of the tools of Riemannian geometry allows to derive a generic algorithm for any kind of data with smooth constraints, which lie therefore on a Riemannian manifold. Stochastic approximations of the Expectation-Maximization algorithm is used to estimate the model parameters in this highly non-linear setting. The method is used to estimate a data-driven model of the progressive impairments of cognitive functions during the onset of Alzheimer's disease. Experimental results show that the model correctly put into correspondence the age at which each individual was diagnosed with the disease, thus validating the fact that it effectively estimated a normative scenario of disease progression. Random effects provide unique insights into the variations in the ordering and timing of the succession of cognitive impairments across different individuals.

The paper proposes a class of mixed-effects models for longitudinal manifold-valued data, as well as a stochastic EM method for a special case. This is applied to real data on Alzheimer's disease progression. Spatiotemporal modelling is a very timely topic. It's a hot topic in medical imaging, and very relevant to longitudinal clinical datasets (as here).Novelty: The methods here are related to those in [8] and [17], as the authors acknowledge. There are important differences, starting with the fact that the present paper applies to any Riemannian manifold, whereas [8] considers only diffeomorphism groups (appropriately, in an analysis of shape). More discussion of the relationship to previous work would be welcome. For example, lines 71-72 are puzzling: "Although the development of statistical models for manifold-valued data is a blooming topic (see [16], [17]), the construction of statistical models for longitudinal data on a manifold remains an open problem." (The title of [16] includes "longitudinal", as does [8].)Also, you say of [8] in Line 75 "the variance of shapes does not depend on time whereas it should adapt with the average scenario of shape changes". I don't understand this statement, and how your scheme does allow time-dependent variance. (Do you mean \epsilon in Eqn (1)?). The particular SAEM method seems somewhat novel to me, but the authors should discuss similarities and differences from [2].I was disappointed that the geometric framework, while starting in a general manifold setting, moved to a product of 1-D manifolds, all with the same metric. I am left wondering whether the paper might have reached a wider audience if presented on R^N with less jargon. On a related point, is the multivariate logistic curves model really necessary? There exists a mapping from (0,1) to \R that transforms the metric in Section 2.4 to the Euclidean one, after which geodesics are straight lines, thus simplifying the exposition and, unless I've missed something, the SAEM algorithm (see line 245 and lines 251 onward).Nonetheless, a general geometric framework can be very valuable. I like the idea of using parallel transport and exponentiation to parametrize the neighbourhood of a curve. There's a "tubular neighbourhood theorem" that says this is really a parametrization, at least if you stay close enough to the original curve; you should find and cite some version of it. The version I recall requires your vector w to be perpendicular to the curve, as you have required, though I presume more general versions exist. I am not convinced by your statements on lines 156-159. First: "The orthogonality condition ensures that a point on the parallel curve wi (, ·) moves at the same pace as in the average trajectory." I don't think that's correct, if "pace" means speed. E.g. if \gamma is the equator of a sphere, then the parallel curves are latitudes, all parametrised by longitude, but with speeds proportional to their radii. The same-pace claim is of course true in a flat metric, as in the motivating example, but orthogonality is not required in that case. Second, on line 159: "vectors w with the same orthogonal projection in (t0) lead to the same geometric parallel curve but with a different time parameterization." I don't think that's correct either (e.g. on a sphere). Nonetheless, I think the desired conclusion, on lines 160-161: "spatial and temporal transformations commute" is correct, by definition.The restriction of the typical disease progression to be a geodesic in \R^N, with the same metric on each dimension, seems quite restrictive. In the example, it means that (as the authors write in Section 2.3) "all biomarkers have on average the same dynamics but shifted in time" [and progressing more or less quickly]. That would seem an unreasonable assumption for many diseases. Nonetheless, it's a reasonable place to start, and has the great convenience of allowing a very low-dimensional parametrization of the mean dynamics. Note that the curve \gamma need not be a geodesic in the general geometric framework.The adapted SAEM algorithm impresses me. Even if it's not actually needed for this example (as I suspect, see above), it seems valuable. Could the authors please clarify its similarities to and differences from citation [2], and comment on its wider applicability?The experiment is interesting and topical. Fig 5 in particular suggests that the model is useful, demonstrating that the estimated time shifts correlate well with the age of conversion to AD, while the rate of progression doesn't.The English is very good but not perfect. Some typos and minor comments are noted below. p1 typo: "fourty"p2, line 89: shall - > should?p2, line 106: N-dim in R^N?? should be R^P for P > =Npp2/3: a citation to some standard reference on Riemannian geometry would be helpful, one that includes the parallel curve concept in Definition 1.p4, line 168: why choose a log-normal distribution? (no criticism, just a request for comment)p4, line 173: why choose a Laplace distribution?p4, line 181: could you choose another letter for \eta_i since \eta is used earlier for the parallel curve?p4, line 181: remove subscript k from \epsilonp5, line 231: where did A go?p5, line 244: "prevents from resorting ot" - > "prevents us from resorting to"p5, line 253: "shall write"? - > "may be written"?p8, line 378: "p_0=" should be followed by "0.3"Citations: capitalize Alzheimer's, EM algorithm, Riemannian ... and many others


Sparse PCA via Bipartite Matchings
We consider the following multi-component sparse PCA problem: given a set of data points, we seek to extract a small number of sparse components with disjoint supports that jointly capture the maximum possible variance. Such components can be computed one by one, repeatedly solving the single-component problem and deflating the input data matrix, but this greedy procedure is suboptimal. We present a novel algorithm for sparse PCA that jointly optimizes multiple disjoint components. The extracted features capture variance that lies within a multiplicative factor arbitrarily close to 1 from the optimal. Our algorithm is combinatorial and computes the desired components by solving multiple instances of the bipartite maximum weight matching problem. Its complexity grows as a low order polynomial in the ambient dimension of the input data, but exponentially in its rank. However, it can be effectively applied on a low-dimensional sketch of the input data. We evaluate our algorithm on real datasets and empirically demonstrate that in many cases it outperforms existing, deflation-based approaches.

I have read the paper "Sparse PCA via Bipartite matching".The paper studies the problem of sparse PCA in the setting of multi-components with disjoint supports. The paper is generally well-written.The main idea is well presented and is generally easy to digest. However, the major problem of the paperis the focus on the case of"disjoint components".In detail,  in equation (2) and the line below, the paper requires all supports of X^{(i)} are disjoint with each other; all the methods theory are built upon such a constraint. This raises serious doubt on how the algorithm can be used (or to be advantageous over existing ones),especiallyfrom a practical perspective.Take gene microarray datafor example (the paper presents some examples on the microarray data),it is believed that the support of different components are monotone: the support of genes differentially expressed between a normal patient and a patient with a malignant tumor is usually a subset of that between a normal patient and a patient with a malicious tumor. In other words, if we have two principal components $\mu_1$ and $\mu_2$, then it is more likely that \[\mathrm{supp}(\mu_1) \subset \mathrm{supp}(\mu_2), \]instead of\[ \mathrm{supp}(\mu_1) \cap \mathrm{supp}(\mu_2)= \emptyset.\] Even from a theoretical viewpoint,assuming the supports are disjoint is rather strong. While I appreciate the paper has made a fair contribution in terms of algorithm (which isnot that significant though, given the existing literature on sparse PCA), I think the paper lacks a good motivation, which largely down-weights the importance of the paper. To patch up such a weakness, I would suggest the authors make a serious discussion on the following problems, especially in methods and theory. \begin{itemize}\item How broad is the model in equation (2) and the line below.\item In case of overlapping supports, how would wedeal with it? If theres isaconvenient extension of the presentedapproach, would the promised advantages continue to exist?  \end{itemize}   Overall, while I like some of ideas inthe paper, I think there are serious drawbacks. Based on these, I would like to recommend rejection with resubmission.


Matrix Completion Under Monotonic Single Index Models
Most recent results in matrix completion assume that the matrix under consideration is low-rank or that the columns are in a union of low-rank subspaces. In real-world settings, however, the linear structure underlying these models is distorted by a (typically unknown) nonlinear transformation. This paper addresses the challenge of matrix completion in the face of such nonlinearities. Given a few observations of a matrix that are obtained by applying a Lipschitz, monotonic function to a low rank matrix, our task is to estimate the remaining unobserved entries. We propose a novel matrix completion method that alternates between lowrank matrix estimation and monotonic function estimation to estimate the missing matrix elements. Mean squared error bounds provide insight into how well the matrix can be estimated based on the size, rank of the matrix and properties of the nonlinear transformation. Empirical results on synthetic and real-world datasets demonstrate the competitiveness of the proposed approach.

This paper proposed extends the single index model in statistics to low-rank matrix estimation. Particularly, the authors proposed to recover the underlying low-rank matrix given the observations after nonlinear transformation. An algorithm based on alternative minimization is proposed to estimate both the transformation function and the underlying low-rank matrix. Theoretical analysis is given for the proposed algorithm. Finally, experiments on both synthetic data and benchmark datasets are conducted to demonstrate the effectiveness of the proposed algorithm in recovering the matrix.  Major comments   1. A quick tip is that the bound in (18) can be greatly simplified. For example, the forth term is dominated by the second term in the setting of this paper.  2. Based on the proof in the appendix, the error bound obtained in Theorem 1 should hold with high probability. The authors should point this out explicitly. 3. As the authors also mention in the paper that the problem being solved is a non-convex problem, a natural question is whether the proposed algorithm will converge to a stationary point. However, this question is not answered in this paper. 4. Only the error bound of the one-step estimator ($T = 1$) is given. However, the error bound of the proposed algorithm is much more important and essential when T is larger than one. Also, as indicated by the experimental part, the one-step estimator is worse than the SVD estimator. In this sense, the analysis is not complete.  5. Line 333, the statement of "For such matrices $\mu_1 = O(), \mu_2 = O()$" is NOT obvious to me.Further justifications are required for rigorous analysis.    6. Line 347, the statement of "If we are given ...." contradicts with the definition of $\tilde{O}$ in Line 306 that there should be a logarithmic term, let alone that a constant is missing.  7. In Line 085, the setup of the problem is that the entries in the matrix are observed with noise; however, in the experimental part, the settings correspond to noiseless observations. The authors should at least add more experiments corresponding to the settings being studied in this paper.    8. The writing and organization of the proof (Appendix) in this paper should be improved significantly. Based on the current writing, it is almost impossible to check the correctness of the theoretical analysis.i) Line 104, it is not clear what A1-A8 refer to.  ii) Line 226 (Appendix), it is not evident what I'_4 is, which leads to confusion over statement in line 233.  iii) Line 191, "Hence, A - Z" ---> "Hence, A - \hat{Z}".  iv) (31) is redundant.


Learning Bayesian Networks with Thousands of Variables
We present a method for learning Bayesian networks from data sets containing thousands of variables without the need for structure constraints. Our approach is made of two parts. The first is a novel algorithm that effectively explores the space of possible parent sets of a node. It guides the exploration towards the most promising parent sets on the basis of an approximated score function that is computed in constant time. The second part is an improvement of an existing ordering-based algorithm for structure optimization. The new algorithm provably achieves a higher score compared to its original formulation. Our novel approach consistently outperforms the state of the art on very large data sets.

The paper describes tricks to scale Bayesian network structure learning to thousands of variables. This is achieved by developing new heuristics for candidate parent set identification and the subsequent order based structure optimization.In general, the paper is clearly written and easy to read. There are issues in editing and style, but the problems do not affect readability (much). The suggested heuristics feel bit ad-hoc, thus the value of the work is eventually judged by empirical evaluation. In this regard, I feel that the work could be better. Notably, the comparisons to simple local greedy searches (that scale well to 10000 variables too) should be included or referred to.In the following, I will give more detailed comments:- Intro, second paragraph: "Usually" - I do not know where this "usually" comes from, but for me it is annoying. It also occurs in many other places in a text, and more often than not it provokes a counter-reaction. Yes, such a two phase strategy has been used before, but to say it is "usually" done this way is a statement that has to be proven right.- Page 2, second paragraph. It is inconsiderate to use citation as a part-of-speech as in [17]. Even for an expert reader, that requires stopping reading and looking for the reference. - Page 2, the BIC formula looks a bit ugly. It would have been better to do $BIC(G)=\sum_{i=1}^n BIC(X_i,\Pi_i)$, and  $$ BIC(X_i,\Pi_i) =...- Page 3, 3.1. The explanation makes the reader suspicious. It is exactly the non-monotonic nature of the effect of combining the parent sets that makes the structure learning hard. How would BIC*  manage with Z=X xor Y (or more generally a parity function). But, well, the experiments will tell.- Page 5, Which data was used to produce Fig 1.- Page 5, while the space of orderings is indeed less that space of DAGS, 1000! is still quite a big number, so calling it "convenient" is a bit of a stretch.- Page 6, 5.1. Sample sizes would have been nice to have too. Maybe n/N in Table 1.- Page 7, last paragraph. Gobnilp is said to be better because it is exact solver. Now, it is exact solver also when n > 500, so how can it all of the sudden be worse? I mean, the explanations as they stand now, are not very convincing. - Page 7. How did you set the structure and parameters for random networks?All in all, it is a rather simple paper and I find it interesting to scale the BN structure learning to 10 000 variables. I really would have liked to see a better empirical comparison with heuristic methods like greedy local search also with big enough sample sizes that allow a big number of parents for the optimal network.


Saliency, Scale and Information: Towards a Unifying Theory
In this paper we present a definition for visual saliency grounded in information theory. This proposal is shown to relate to a variety of classic research contributions in scale-space theory, interest point detection, bilateral filtering, and to existing models of visual saliency. Based on the proposed definition of visual saliency, we demonstrate results competitive with the state-of-the art for both prediction of human fixations, and segmentation of salient objects. We also characterize different properties of this model including robustness to image transformations, and extension to a wide range of other data types with 3D mesh models serving as an example. Finally, we relate this proposal more generally to the role of saliency computation in visual information processing and draw connections to putative mechanisms for saliency computation in human vision.

Summary:The paper proposes a definition of visual saliency which is based onMaxima in Information Scale-Space (MISS). Connections to scale-space theory and information theory are explained, as well as to some other works. Several experiments are reported, including on fixation and salient objects in 2D, and 3D salient features of 3D meshes.  Quality: The paper is well written, and it seems technically correct.Connections between information theory and saliency are interesting (and to some extent have been explored in the past by others, although in somewhat different formulations), and combining these with scale-space theory makes a lot of sense (and, again, to some extent, has also been explored before; e.g., see below).One problem with the quality of the paper is that the technical/mathematical contribution is fairly small, and pretty much boils down to a definition of a saliency measure. Clarity:The paper is well written.Originality:It is great the authors point out the connections to the classic works on scale-space theory, and tie together scale-space theory and information theory in the context of visual saliency. There is a paper, however, by Toews and Wells ("A mutual-information scale-space for image feature detection and feature-based classification of volumetric brain images", CVPRW, 2010) which seems to diminish *some* of the novelty of the present work. May the authors please clarify the relation to (and differences from) from Toews and Wells? Significance: This paper may have some positive impact on a subset of the NIPS community.  Remark: it is always a pleasure (for me) to see a deep net fails (Figure 5 -- where the results obtained by the proposed method are clearly better than those obtained by the deep-net approach). However, perhaps the authors should add at least one counterexample? Surely there was at least one image where the SALICON output was better than the proposed method's output...


Basis Refinement Strategies for Linear Value Function Approximation in MDPs
We provide a theoretical framework for analyzing basis function construction for linear value function approximation in Markov Decision Processes (MDPs). We show that important existing methods, such as Krylov bases and Bellman-errorbased methods are a special case of the general framework we develop. We provide a general algorithmic framework for computing basis function refinements which "respect" the dynamics of the environment, and we derive approximation error bounds that apply for any algorithm respecting this general framework. We also show how, using ideas related to bisimulation metrics, one can translate basis refinement into a process of finding "prototypes" that are diverse enough to represent the given MDP.

This paper deals with basis construction for VFA, based on the concepts of equivalence relations and bisimulation metrics. The authors define a specific equivalence relation (consistency resp. to a subspace) and a concept of refinement and show (Th. 3.1) that starting from a reward equivalence relation, a sequence of refined bases (such that the cardinality of the newly generated basis is equal to the cardinality of the quotient set corresponding to the equivalence relation based on the span of the previous basis) will provide a (strong) guarantee regarding the quality of the optimal value function associated to this base. Then, they relax the introduced equivalence relation to take into account locality, and show a property on the link between local and global refinements. Next, they discuss a convergence criterion for this refinement principle based on bissimulation (equivalence and metric). Lastly, they extend the proposed refinement strategy by alternating metric learning and base learning (through prototypes).The following detailed comments follow the paper (they are not ordered by importance). * l.81: s_i,a_i -> s_i,r_i ? * l.86: it may be useful to define the concepts of equivalence relation, equivalence class and quotient set (as they are a core of the paper) * l.116: s1 sim s2 -> s sim s' * l.116: precise that Pa(s)(C) is the measure of the set C * l.122: a missing absolute value in the def. of Omega(d)? * l.132: what is V^*_Delta? The least-squares fixed-point approximation (for the subspace Delta(sim)) of the optimal policy pi_*? How can this be computed? * l.157: beta is not defined and the notation Gamma is used afterwards * l.161-165: it is not clear how the refining fix inconsistencies in the dynamics (eg, if f(s) neq f(s'), take varphi = f, the subspace does not change) * l.173: it should be said that sim Sigma is sim span(Sigma) (even if rather clear from the context) * Th.3.1: how could be constructed the refinement? (the authors state that sec. 5 provides an answer, but see latter comments) Given the condition on the size of the refinement, how will the sequence of subspaces grows? Isn't it possible that a few refinements could lead to the whole space F_S (in which case the result is trivial)? * l.209: what is the (pseudo)-metric 0? the metric that associates 0 to any pair of states? * l.215: the notation P^a(s)|C is not defined (consequently, the rest of the proof is obscure) * l.231: where does the inequality in Eq. 3 come from? what is the supremum norm for the norm (entrywise, induced, other)? * l.233: what is V^*_n? is it V^*_Gamma_n? * l.235: in Eq.4, gamma/(1-gamma)^2 -> 1/(1-gamma)^2 ? * Prop.3.1: how is initialized the sequence? Above all, what is Gamma_(0,eta)? As it is not defined, it is hard to understand the stated result. * Sec.3.2: it is hard to see how Krylov bases/state aggregation relate to the stated result (apart from the addressed problem, that is basis construction). Do they satisfy the assumptions of Th.3.1? * l.314: Section 3.1 -> Section 3.2 * l.330: P and M are not defined * l.389: what is a supremum over metrics? * Def.4.1: it is hard to see the implications of this def., especially how it relates to a convergence criterion for subspaces refinement * Sec.5: the proposed approach is not clear at all. Providing a pseudo-code might help (and maybe instantiating the general approach too; for example how to find a cover over S, covering with what?). * l.367: what S subset Q means? why assuming it? * l.374: hat(J)n is not defined * l.376: iota(kappa) should be iota(kappa)(s). By the way, the metric D(Gamma) takes too states as argument, how can it takes a prototype kappa as the first argument? * l.423: significantly chapter?


Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
The goal of precipitation nowcasting is to predict the future rainfall intensity in a local region over a relatively short period of time. Very few previous studies have examined this crucial and challenging weather forecasting problem from the machine learning perspective. In this paper, we formulate precipitation nowcasting as a spatiotemporal sequence forecasting problem in which both the input and the prediction target are spatiotemporal sequences. By extending the fully connected LSTM (FC-LSTM) to have convolutional structures in both the input-to-state and state-to-state transitions, we propose the convolutional LSTM (ConvLSTM) and use it to build an end-to-end trainable model for the precipitation nowcasting problem. Experiments show that our ConvLSTM network captures spatiotemporal correlations better and consistently outperforms FC-LSTM and the state-of-theart operational ROVER algorithm for precipitation nowcasting.

The paper describes an extension to the fully connected long short-term memory models for the purpose of prediction of doppler radar echo for a region over a short period of time given the recent doppler radar echo images.The proposed framework extends LSTMs to handle the temporal evolution of spatial fields.For the experiments, the paper considers two set ups, an artificial one based on the movements of handwritten digits, and a data set of Doppler radar images for unspecified region and time.Quality: The proposed extension ConvLSTM appears to be mathematically sound and appropriate for the modeling of regularly timed spatio-temporal data.The paper provides only a basic formulation of the framework; there are few details on the training of the model -- it is assumed that the reader is familiar with how FC-LSTMs are trained.How does the training of ConvLSTM differ?There are too few details provided about the precipitation data to judge whether the set up is appropriate.It appears to be somewhat artificial to me in the following aspects: (1) storms can be fast moving (e.g., Midwest of the United States), so including the data only for the subregion of prediction may ignore the crucial information about the storm if it falls outside of the prediction area; (2) only days with precipitation are being used as a part of the data -- this appears unreasonable for a real application, (3) other covariates (e.g., time of day) are not taken into an account.Additionally, to demonstrate an impact for the application, it would be appropriate to compare the predictive ability not just to other machine learning methods, but the standard methods in the application area, numerical weather prediction models (https://www.ncdc.noaa.gov/data-access/model-data/model-datasets/numerical-weather-prediction).Clarity: The paper tries to cover quite a bit of ground, unfortunately, at the expense of clarity.The details on the proposed model are fairly bare.I would suggest to focus either on the application (and in this case, skip the dancing digits while elaborating more on the primary application) or on the methodology (and provide more details on the property of the new model at the expense of the details on the nowcasting application).Originality: There are original contribution to the modeling framework and in applying it to the short-term precipitation forecasting.Personally, I am glad to read that deep learning techniques are reaching precipitation modeling research area.Significance: I do not have the expertise to judge whether the incremental leap between FC-LSTM and ConvLSTM is significant.For the nowcasting application, while the proposed framework can be a useful intermediate step, due to the reservations mentioned under Quality section, I doubt it is close to solving the problem.Minor: - Please check equation (3); I think some of the Hadamard products should actually be convolution operators. - Page 1, The acronym for numerical weather forecasting is NWF.Did you mean numerical weather prediction? - Footnote on page 1, "plan"->"plane"?


Convolutional Spike-triggered Covariance Analysis for Neural Subunit Models
Subunit models provide a powerful yet parsimonious description of neural responses to complex stimuli. They are defined by a cascade of two linear-nonlinear (LN) stages, with the first stage defined by a linear convolution with one or more filters and common point nonlinearity, and the second by pooling weights and an output nonlinearity. Recent interest in such models has surged due to their biological plausibility and accuracy for characterizing early sensory responses. However, fitting poses a difficult computational challenge due to the expense of evaluating the log-likelihood and the ubiquity of local optima. Here we address this problem by providing a theoretical connection between spike-triggered covariance analysis and nonlinear subunit models. Specifically, we show that a "convolutional" decomposition of a spike-triggered average (STA) and covariance (STC) matrix provides an asymptotically efficient estimator for class of quadratic subunit models. We establish theoretical conditions for identifiability of the subunit and pooling weights, and show that our estimator performs well even in cases of model mismatch. Finally, we analyze neural data from macaque primary visual cortex and show that our moment-based estimator outperforms a highly regularized generalized quadratic model (GQM), and achieves nearly the same prediction performance as the full maximum-likelihood estimator, yet at substantially lower cost.

The authors present a moment-based method for estimating the nonlinear subunits of neurons in sensory systems. They begin by giving a formal definition of a convolutional neural subunit model and then proceed to show three methods for estimating the parameters of such a model. Finally, then present theorems for the identifyability of neural subunits, and show good performance in experiments with simulated and biological data.The paper is clear and well written. It is a novel approach to estimation of neural subunit models. It provides a nice foundation for this class of work and is therefore significant to the field. There are no major concerns.Minor questions: - Besides a decrease in MSE, what are the functional consequences for estimating the parameters of a model for data that is not generated with a quadratic subunit nonlinearity? - why does runtime go down with sample size in figure 2b? - can you clarify the following sentence on line 360? "All MLEs for each underlying model are equipped with the corresponding nonlinearity thus provides a baseline" - the binary white noise stimuli used in the biological experiment do not satisfy the Gaussian assumption. How does this affect the results? - can you explain the shape and sign of w in figure 4a?Also, there is a typo in 'frequency' in line 196.


Information-theoretic lower bounds for convex optimization with erroneous oracles
We consider the problem of optimizing convex and concave functions with access to an erroneous zeroth-order oracle. In particular, for a given function x → f (x) we consider optimization when one is given access to absolute error oracles that return values in [f (x) − , f (x) + ] or relative error oracles that return value in
, for some > 0. We show stark information theoretic impossibility results for minimizing convex functions and maximizing concave functions over polytopes in this model.

Added post-rebuttal: Thanks for addressing my technical concerns, and I've raised my score to 6. Note that the \delta,\epsilon issue still seems a bit weird (for example, theorem 4.2, why not just take delta=0.5, getting the strongest possible statement?).============================================The paper studies lower bounds for zero-order optimization, where the returned function values have some arbitrary bounded error. Previous results provided algorithms when the error scales down linearly in the dimension n. In contrast, here it is shown that if the error scales more slowly than n^{-1/2}, then no non-trivial optimization is possible. This is shown in several settings, considering both absolute and relative errors, and for several different optimization domains. The proof technique is based on constructing two functions which agree almost everywhere on the domain, yet have quite different optima. I am a bit undecided regarding this paper. On one hand, the basic question is scientifically interesting, and the answer is somewhat surprising at first, and potentially useful. On the other hand, there are several weaknesses:- Beyond the first interesting result (theorem 3.1), the other results are variations of the same idea for various specific domains (whose relevance for zero-order optimization is unclear), and I didn't feel they provide anything conceptually new. I think the paper would have been stronger if it contained more genuinely different results. For example, is it possible to generalize the impossibility results here to general families of domains? What is the error in the oracle values is between n^{-1/2} and n^{-1} (at which point existing algorithms do work)? What if the oracle error isn't arbitrary but has some structure?- The connection of the paper to machine learning is somewhat tenuous (in previous papers, which considered a noisy oracle, one can at least argue that such an oracle reflects uncertainty due to data sampling).- Although I believe the results, the proofs are sloppy at places and some parts need to be fixed (see technical comments below). - The text is wider than the required format (maybe because of an erroneous use of \fullpage in the latex?) This means that the paper will require some non-trivial editing to get to the page limit.  Specific Comments ----------------- - All the proofs rely on extending a Chernoff bound for a fixed query to a sequence of several queries, using a union bound. However, after the first query, the other queries are no longer fixed, but rather conditioned on previous queries. So, one cannot apply a Chernoff bound as-is, and a more delicate argument is required.- In theorem 3.1, one should also require delta < 1, no? Otherwise the proof doesn't seem to work. Similarly, in theorem 4.2, I think delta must be < 0.5 (otherwise, in the proof, f and g would be the same function).- In all the proofs, there is an implicit assumption that n^{delta} and such numbers are integers (e.g. splitting [n] to sets of this size), which is not necessarily the case.- In the proof of theorem 4.2, the definition of the polytope domain is missing (and is apparently not the same as in theorem 4.1, since the optimal values of f and g would be different than those stated in the proof). - In the proof of theorem 3.1, the fact that the function value is the sum of negatively correlated random variables needs to be more carefully justified, either by a reference or by explicitly showing why they satisfy the definition of negative correlation from section 2.- The results of section 4 appear to be stronger the larger we pick epsilon,delta. So, why not show the result for some large constant epsilon,delta.?- Section 5 is extremely short and appears to be just an advertisement to something which appears in supplementary material. This makes the paper not self-contained. I would recommend either providing some formal result (possibly without the proof), or dropping this section. - line 319: "We again we select"


Bounding the Cost of Search-Based Lifted Inference
Recently, there has been growing interest in systematic search-based and importance sampling-based lifted inference algorithms for statistical relational models (SRMs). These lifted algorithms achieve significant complexity reductions over their propositional counterparts by using lifting rules that leverage symmetries in the relational representation. One drawback of these algorithms is that they use an inference-blind representation of the search space, which makes it difficult to efficiently pre-compute tight upper bounds on the exact cost of inference without running the algorithm to completion. In this paper, we present a principled approach to address this problem. We introduce a lifted analogue of the propositional And/Or search space framework, which we call a lifted And/Or schematic. Given a schematic-based representation of an SRM, we show how to efficiently compute a tight upper bound on the time and space cost of exact inference from a current assignment and the remaining schematic. We show how our bounding method can be used within a lifted importance sampling algorithm, in order to perform effective Rao-Blackwellisation, and demonstrate experimentally that the Rao-Blackwellised version of the algorithm yields more accurate estimates on several real-world datasets.

The paper introduces lifted and/or schematics, which are an abstract representation of the PTP search space. Analyzing the schematic lets us quantify the size of the search space. This is of independent theoretical interest (although the theory is not quite there yet to provide deep insights), and is practically useful in the context of Rao-Blackwellized importance sampling.I believe this type of work is important; we still lack understanding of lifted inference algorithms. The proposed schematics are not really simple enough to provide a whole lot of insight, but it's a move in the right direction. The presentation is okay, although I cannot verify many of the technical points with the time available to me.Given that this is a theoretical paper, I wonder how powerful the framework is in terms of liftability. Can these schematics capture all known classes of models that are domain-liftable?The paper misses the opportunity to relate to other similar techniques.How are dtrees (used in recursive conditioning and knowledge compilation) related to and/or schematics?How are first-order NNF circuits related to lifted and/or schematics? They look very similar, with counting nodes having a single child, which induces a summation when evaluated, and so on. There is some work on the complexity of evaluating partition functions from first-order NNF circuits, where the result is that it is exponential in the number of nested counting operations (existential quantifiers). This result seems related to the analysis in Section 4.How does this relate to the first-order decomposition trees of Taghipour et al.? The footnote on page 3 says that these are not as tight: I would expect more details here (and perhaps an example of where the complexity differs).I like the importance sampling algorithm that uses the theory in a meaningful way. It's very convincing. I know these experiments are preliminary, but I would still like more details on the setup. For example, I do not believe that 10% evidence on webkb leaves you with any symmetries to exploit. I suspect that the evidence predicates were chosen to maximize the remaining symmetries (only evidence on page class, not on links?). It would be good to clarify.Typos: The theta_i on line 105 should be a t_i. With exchangeable variables on line 171, you mean exchangeable constants (the random variables after shattering are not fully exchangeable in general). Line 212: of depends.


Natural Neural Networks
We introduce Natural Neural Networks, a novel family of algorithms that speed up convergence by adapting their internal representation during training to improve conditioning of the Fisher matrix. In particular, we show a specific example that employs a simple and efficient reparametrization of the neural network weights by implicitly whitening the representation obtained at each layer, while preserving the feed-forward computation of the network. Such networks can be trained efficiently via the proposed Projected Natural Gradient Descent algorithm (PRONG), which amortizes the cost of these reparametrizations over many parameter updates and is closely related to the Mirror Descent online learning algorithm. We highlight the benefits of our method on both unsupervised and supervised learning tasks, and showcase its scalability by training on the large-scale ImageNet Challenge dataset.

This paper develops a neural networks optimization method based on the idea of optimizing a version of the network where the values of the units at each layer are whitened and centered.This is reminiscent of centering approaches, which only do the centering.The method is presented as a type of approximate natural gradient descent, and competitive/superior performance compared to methods like SGD and Batch Normalization is demonstrated on various deep neural network training problems.The paper is reasonably well written and the approximation approach seems interesting, although quite similar to several methods that have appeared recently (which are cited).The empirical comparisons seems mostly solid, although I'm not convinced that they are as fair as they could be.The approximation of the natural gradient used in the approach is not given any theoretical justification, although judging by the experiments it seems to be a reasonable one in practice.   L 35: what do you mean here by "probabilistic manifold" and "constant progress"?Surely the amount of progress per iteration of natural gradient descent will change as optimization proceeds.L 44: "do not take into account the smooth change".Actually I think some of these methods initialize their inner iterations from previous solutions so they would benefit from a slowly changing FisherL 120:Assuming this kind of independence seems like a very severe approximation to make, as these quantities are obviously closely related to each other.Do you have any justification for making this approximation?L 136:PNGD seems like a poor choice for a name since it sounds like it could describe any method combining ideas of natural gradient descent with projected gradient descent, instead of the very specific approximations that you are using here (which are also neural-net specific).L 152:I don't see why this has anything obviously to do with a trust-region.It should be spelled out in more detail.L 191:I found this paragraph very hard to follow.I think you should be more explicit about what you mean here by this diagonal rescaling of U, providing an equation etc.Are the rows of U being rescaled?In this case wouldn't you then have to rescaling the columns of V to preserve the feed-forward computation?Are the columns being rescaled (which is the interpretation I'm going with)?Then I don't see how any simple manipulation of V would preserve the computation.One could recompute V as W D^-1 U^-1, whenever U is replaced with U D, but this certainly doesn't correspond to a row scaling of V.L 212: is the * supposed to be after \nabla like that?L 215: Is it supposed to be \Omega^(t+1) in brackets here?L 235: I don't understand what it means to use a "whitened mirror map".Won't defining \psi completely change the kind of update you compute?L 238: Won't the updates made to \Omega involve multiplying by inverses of matrices too?How will this make things more efficient than the naive approach?L 256: I think the square root connection is entirely spurious.If what you do is equivalent to a form of natural gradient descent, then the updates, as considered in the original theta space, won't depend on square roots in any way.This is much unlike the case with RMSprop.L 260: I don't believe that the K-FAC approach uses any low-rank approximations.L 310: An improved condition number doesn't imply that the matrix is close to diagonal.Figure 2b) doesn't look particularly diagonal to me.L 332: Why do plots c) and d) both have validation error on the y axis?L 350: Sutskever et al (2013) showed that momentum is very important when optimizing these kinds of deep autoencoders and that the coefficient matters a lot.Choosing a fixed value from {0,0.9} is probably causing SGD with momentum to underperform here.L 384: What do you mean by extra perturbations?Where is this coming from?L 416: As far as I can tell, K-FAC does something like this but without the need for skip-connections, so it's not obvious that they should be needed.


Regularization Path of Cross-Validation Error Lower Bounds
Careful tuning of a regularization parameter is indispensable in many machine learning tasks because it has a significant impact on generalization performances. Nevertheless, current practice of regularization parameter tuning is more of an art than a science, e.g., it is hard to tell how many grid-points would be needed in cross-validation (CV) for obtaining a solution with sufficiently small CV error. In this paper we propose a novel framework for computing a lower bound of the CV errors as a function of the regularization parameter, which we call regularization path of CV error lower bounds. The proposed framework can be used for providing a theoretical approximation guarantee on a set of solutions in the sense that how far the CV error of the current best solution could be away from best possible CV error in the entire range of the regularization parameters. Our numerical experiments demonstrate that a theoretically guaranteed choice of a regularization parameter in the above sense is possible with reasonable computational costs.

A common task in machine learning is to learn the regularization parameter of a model. Usually this is done using cross-validation. The main result of the paper is that is presents a new way to analytically compute bounds on validation error. First, a lower and an upper bound for wx_i is computed for each validation instance x_i (w is the parameter of the model) as a function of the regularization parameter, given a solution for a different regularization parameter. Then, the scores for validation instances are combined to yield the bounds on the validation error. A central result in the paper is Lemma 1, that shows how compute the bounds for wx_i. The proof is not given in the paper, but the reader is referred to the Appendix. Giving intuition of this core result in the main text would be very important for the reader to be able to understand the idea without going to the Appendix, even if it's entirely reasonably to omit the full proof from the main text. The proof (in the appendix) seems to rely heavily on the convexity of the regularized loss function, limiting the usability (although being more general than presented before). Also, L2 prior on the parameter is used. Furthermore, the paper only deals with tuning a univariate hyper-parameter. The results appear correct, though I did not check the proofs in detail. The text is clearly written (though see the remark above). The experiments show that the regularization parameter can be optimized efficiently by assuming a reasonably tight error bound and, importantly, that the error bound is available in the first place. The topic seems relevant for the NIPS community, and the results interesting, though potentially too specific to raise wide interest.


Convergence Analysis of Prediction Markets via Randomized Subspace Descent
Prediction markets are economic mechanisms for aggregating information about future events through sequential interactions with traders. The pricing mechanisms in these markets are known to be related to optimization algorithms in machine learning and through these connections we have some understanding of how equilibrium market prices relate to the beliefs of the traders in a market. However, little is known about rates and guarantees for the convergence of these sequential mechanisms, and two recent papers cite this as an important open question.
In this paper we show how some previously studied prediction market trading models can be understood as a natural generalization of randomized coordinate descent which we call randomized subspace descent (RSD). We establish convergence rates for RSD and leverage them to prove rates for the two prediction market models above, answering the open questions. Our results extend beyond standard centralized markets to arbitrary trade networks.

Summary of paper: The authors consider the rate of convergence to price equilibrium in prediction markets.They exploit techniques from convex analysis and programming to derive new such bounds for a variety of settings.In more detail, they consider a potential-based market maker and a number of traders whose preferences are modeled by a risk measure. The market maker can be interpreted as a constant-risk agent.All players have initial positions in k securities.Trading dynamics can be naturally interpreted as local rebalancing the positions of a subset of players to minimize their total risk (subject to the sum of their positions remaining the same). Thus the question of convergence rate is closely related to the question of convergence of gradient descent, where at each iteration only a subset of coordinates are allowed to be updated.The authors adapt techniques from the latter problems to prove convergence rate bounds for the former problems. Quality: I am in favor of further developing the connections between convex optimization and prediction market design and analysis.The question of convergence in prediction markets is clearly interesting and, perhaps surprisingly, does not seem very well understood yet.My one complaint about the paper is that the exact technical statement could be more clearly stated and discussed.For example, in Theorem 3, it seems that there is a lot of detail hidden within the "constant" R(x^0).Specifically, this depends not only on x^0, but also on the details of the projections (the \Pi_i's) and how "overlapping" they are.In the proof of Theorem 3 in the supplementary, the matrix A (used in the norm in (10) and subsequently) is undefined.Apparently, going by Section D.2, the devil is largely in the details of this A and its properties.The submission would benefit from a more transparent discussion in the main body about when the convergence rate is good and when it is not.Clarity: The writing is usually good, though see technical comments above.Originality: As far as I know, the connection between the convergence analysis of gradient descent variations and prediction markets is original.Significance: I think this is a significant contribution to the literature on prediction markets.


Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
State-of-the-art object detection networks depend on region proposal algorithms to hypothesize object locations. Advances like SPPnet [7] and Fast R-CNN [5] have reduced the running time of these detection networks, exposing region proposal computation as a bottleneck. In this work, we introduce a Region Proposal Network (RPN) that shares full-image convolutional features with the detection network, thus enabling nearly cost-free region proposals. An RPN is a fully-convolutional network that simultaneously predicts object bounds and objectness scores at each position. RPNs are trained end-to-end to generate highquality region proposals, which are used by Fast R-CNN for detection. With a simple alternating optimization, RPN and Fast R-CNN can be trained to share convolutional features. For the very deep VGG-16 model [19], our detection system has a frame rate of 5fps (including all steps) on a GPU, while achieving state-of-the-art object detection accuracy on PASCAL VOC 2007 (73.2% mAP) and 2012 (70.4% mAP) using 300 proposals per image. Code is available at https://github.com/ShaoqingRen/faster_rcnn.

The paper presents a region proposal network (RPN) which shares the conv features with R-CNN and includes two fully connected layers that predicts object bounds and objectness scores at each position. RPN is heuristically combined with Fast R-CNN for object detection and achieves state-of-the-art performance on VOC07 and VOC12. The running time of generating region proposals is greatly reduced since RPN shares the conv layers with the detection network.Pros: - Smart idea on sharing conv layers between RPN and R-CNN which greatly reduced the run-time of object proposals.- State-of-the-art performance on object detection.- Well-written paper.Cons:- The training of RPN and fast R-CNN are weakly combined by alternating optimization and there is no convergence proof.- RPN can be used independently as a region proposal method. However, it doesn't seem to outperform selective search.


A Fast, Universal Algorithm to Learn Parametric Nonlinear Embeddings
Nonlinear embedding algorithms such as stochastic neighbor embedding do dimensionality reduction by optimizing an objective function involving similarities between pairs of input patterns. The result is a low-dimensional projection of each input pattern. A common way to define an out-of-sample mapping is to optimize the objective directly over a parametric mapping of the inputs, such as a neural net. This can be done using the chain rule and a nonlinear optimizer, but is very slow, because the objective involves a quadratic number of terms each dependent on the entire mapping's parameters. Using the method of auxiliary coordinates, we derive a training algorithm that works by alternating steps that train an auxiliary embedding with steps that train the mapping. This has two advantages: 1) The algorithm is universal in that a specific learning algorithm for any choice of embedding and mapping can be constructed by simply reusing existing algorithms for the embedding and for the mapping. A user can then try possible mappings and embeddings with less effort. 2) The algorithm is fast, and it can reuse N -body methods developed for nonlinear embeddings, yielding linear-time iterations.

Note 1 - this is a 'light review'. Note 2 - I was not able to open the supplementary file, the provided archive .gz did not contain any readable file type. The paper presents an optimization approach for parametric out-of-sample extension of nonlinear dimensionality reduction algorithms.Solutions are constrained a desired parametric family whose parameters are optimized based on an auxiliary coordinate approach, alternatively optimizing a regularized loss with respect to auxiliary coordinates and then the mapping as a regression to the coordinates.Experiments on three data sets show that the method achieves solutions about equal in quality to direct optimization of parametric embeddings but is faster. Another advantage is that it does not require separate derivation of optimization equations for each loss + parametric family combination.  Evaluation:The idea is good and out of sample extension is a common problem in nonlinear dimensionality reduction. The experiments are somewhat brief as they are only on COIL, swissroll and MNIST data; the supplementary file was said to contain more experiments but I was unable to open it.It is hard to evaluate the contribution. On one hand the method answers a clear need. On the other hand, the method seems mostly an adaptation of the principle in ref. 6 from the multilayer network case in that paper to a general loss function setting.  Details:* It is stated "Directly fitting F to (Y,X) is inelegant, since F is unrelated to the embedding, and may not work well if the mapping cannot model well the data" but then state "Alternating optimization ... trains an auxiliary embedding with a 'regularization' term, and ... trains the mapping by solving a regression". Both approaches use a regressor, the difference is in the use of regularization during the alternating optimization; it would be useful to clarify this.* "if manually simplified and coded, the gradient can actually be computed in O(N2L + N |F|)": this would seem to require some proof to show this holds in a general case; the elastic embedding is only a special case.* "Deriving, debugging and coding the gradient of P for a nonlinear F is cumbersome.": how cumbersome is this? If I understand correctly, many nonlinear dimensionality reduction methods are optimized by gradient algorithms anyway, so the gradient with respect to data points is likely known, all that remains is the right-hand term in the chain rule (gradient of coordinates with respect to the mapping). Do you mean that simplification is the cumbersome part?* "each term depends only on 2 projections (zn and zm", what are zn and zm?* "it is even possible to use something like a regression tree for F, which is not differentiable, and so the PE objective function is not differentiable either": does such a non-differentiable F still satisfy the convergence conditions mentioned below eq. (5)?


Bayesian Optimization with Exponential Convergence
This paper presents a Bayesian optimization method with exponential convergence without the need of auxiliary optimization and without the δ-cover sampling. Most Bayesian optimization methods require auxiliary optimization: an additional non-convex global optimization problem, which can be time-consuming and hard to implement in practice. Also, the existing Bayesian optimization method with exponential convergence [1] requires access to the δ-cover sampling, which was considered to be impractical [1,2]. Our approach eliminates both requirements and achieves an exponential convergence rate.

The paper presents a variant of Bayesian Global Optimization where future samples are generated through a space/interval partitioning algorithm.Using UCB, this avoids the (internal) global optimization problem of how to choose the next sample.Theoretical analysis shows that the approach yields theoretical improvements (exponential regret), thereby improving over previous work.The algorithm is clearly described and explained/illustrated with an example.The paper makes a novel contribution and is overall clearly presented.I have one reservation with this paper.The experimental results seem to be only for 1D test functions (not explicitly stated).I suspect that the interval partitioning approach does not scale well to higher dimensions (hyperrectangles) because the "resolution" would be required to grow exponentially.The approach is related to the well-known DIRECT algorithm, which is known to suffer badly when the dimensionality of the problem increases.I think something at least needs to be said about this in the paper.It does not change the theoretical contribution but is clearly significant for any practical purposes.Minor comments:p.2, UCB is considered "for brevity".Does this mean you could do something with expected improvement, for example?I got the feeling it had to be UCB.p.3 "...we simultaneously conduct global and local searches based on all the candidates of the bounds."I couldn't understand this statement.p.4 "At n=16, the far right...but no function evaluation occurs."Can you say why for clarity?


Deep Visual Analogy-Making
In addition to identifying the content within a single image, relating images and generating related images are critical tasks for image understanding. Recently, deep convolutional networks have yielded breakthroughs in predicting image labels, annotations and captions, but have only just begun to be used for generating high-quality images. In this paper we develop a novel deep network trained end-to-end to perform visual analogy making, which is the task of transforming a query image according to an example pair of related images. Solving this problem requires both accurately recognizing a visual relationship and generating a transformed query image accordingly. Inspired by recent advances in language modeling, we propose to solve visual analogies by learning to map images to a neural embedding in which analogical reasoning is simple, such as by vector subtraction and addition. In experiments, our model effectively models visual analogies on several datasets: 2D shapes, animated video game sprites, and 3D car models.

This paper proposes a method to find analogical images using given query images. Specifically, given four images a, b, c, and d, we would like to find a transformed version of c such that it is similar to the transformation from a to b when applied to d. This is an interesting problem and should be of interest to image recognition community in general because if images could be represented in a space (vectors in the proposed method), then such translation invariant representations could simplify the task of recognizing images thereby improving recognition accuracies.Although using analogical methods for image recognition is a relatively new topic in the visual domain, it has a well established history in NLP. For example, SAT word analogy questions have been used to evaluate the performance of analogy detection methods. See @article{Turney_CL,   Author = {P.D. Turney},   Journal = {Computational Linguistics},   Number = {3},   Pages = {379--416},   Title = {Similarity of semantic relations},   Volume = {32},   Year = {2006}} for an overview. Both supervised and unsupervised methods have been proposed for this purpose. Some background description of such prior work on analogy detection will give a good context to the current paper.Moreover, some work has been done in the visual domain studying embeddings that preserve analogical properties in images. @inproceedings{Hwang:ICML:2013,   Author = {Sung Ju Hwang and Kristen Grauman and Fei Sha},   Booktitle = {ICML'13},   Title = {Analogy-preserving Semantic Embedding for Visual Object Categorization},   Year = {2013}}Sec 3.1 argues that sampling from co-occurring contexts results in word embeddings that demonstrate analogical properties when the vector difference is considered. However, this claim is not clear to me. Why would predicting word co-occurrence result in word representations that demonstrate analogies? It is true empirically that this is the case but why predicting co-occurrences should help?In the proposed method this is by design. In other words, the objective functions are designed in such a way to minimize the loss between the target image d and its construction via a transform on a, b, and c. There is a recent work that learns semantic representations for words by predicting analogies directly in similar spirit to the proposed method. Embedding semantic relations into Word Representations IJCAI 2015 arxiv paper.In Algo 1, once you create N candidate images how would you select the best one for d? Do you evaluate the objective function for each candidate and select the best one? A related question is how to decide N? Does it depend on the difference between a and b?One weakness of the evaluation section is that all evaluations are limited to different baselines/settings within the proposed method. It would have been good if there was some comparison against prior work.


Online Prediction at the Limit of Zero Temperature
We design an online algorithm to classify the vertices of a graph. Underpinning the algorithm is the probability distribution of an Ising model isomorphic to the graph. Each classification is based on predicting the label with maximum marginal probability in the limit of zero-temperature with respect to the labels and vertices seen so far. Computing these classifications is unfortunately based on a #P -complete problem. This motivates us to develop an algorithm for which we give a sequential guarantee in the online mistake bound framework. Our algorithm is optimal when the graph is a tree matching the prior results in [1]. For a general graph, the algorithm exploits the additional connectivity over a tree to provide a per-cluster bound. The algorithm is efficient, as the cumulative time to sequentially predict all of the vertices of the graph is quadratic in the size of the graph.

Summary: -------- The paper studies prediction of labels on a graph, where the nodes are data points, the edges indicate similarity in label, and getting different labels on the sides of an edge is penalized. The authors analyze the problem in an online setting, and provide a general mistake bound (Thm. 4) that holds for any algorithm that has three properties: monotonicity (observing a label can only result in more predictions of that label), permutation-invariance (order of observations does not matter), and a "Markov" property (predictions on a node are influenced only by its immediate connections). Next, the authors provide two prediction strategies that satisfy these properties (Thm. 9), one of which (0-Ising) is computationally inefficient while the other (longest-path) has polynomial running time in the size of the graph. Finally, the authors show a per-cluster mistake bound for these two strategies (Thm. 10) that, combined with the general mistake bound of Thm. 4, results in a concrete bound for these two algorithms. The new bound retrieves the previous optimal bound on trees, while it still gives meaningful (in cases improved) bounds when the graph is not a tree.Quality and Clarity: -------------------- Generally, the paper is well-written: the setting, definitions, and logical steps are laid out and explained clearly, which makes the paper easy to read even for those who are not directly working in this area. There are some problems to fix, which I have mentioned at the end.The theory seems to work, although I have not verified all the proofs in detail.Originality and Significance: ----------------------------- The contribution that the paper lays out seems interesting and I think it will be useful for the community. However, I am not working in this area directly, so I leave a deeper comparison with the related work to other reviewers that are more familiar with the previous work in this particular area.Other issues: -------------- It is not specified what is the format of the max-flow returned in step 2 of Figure 1, which propagates to the definition of graph $$I$$ in step 3. In particular, in Figure 2(c), has 1->3->2 been in the max flow? If no, then why has 11->10->12 been there? If yes, then why are the edges 1-3 and 3-2 still in I? - On page 3, paragraph 2, line 7, please fix the definition of the set of edges of the quotient graph. - Perhaps the first sentence of Sec. 2 needs to be moved to the first page to become the first sentence of the last paragraph on that page. Then I think the phrase "this graph" on the third-to-last line on page 1 will have a close-by reference. - perhaps the references [3] and [4] on line 4 of the first paragraph are swapped.


Lifted Symmetry Detection and Breaking for MAP Inference
Symmetry breaking is a technique for speeding up propositional satisfiability testing by adding constraints to the theory that restrict the search space while preserving satisfiability. In this work, we extend symmetry breaking to the problem of model finding in weighted and unweighted relational theories, a class of problems that includes MAP inference in Markov Logic and similar statistical-relational languages. We introduce term symmetries, which are induced by an evidence set and extend to symmetries over a relational theory. We provide the important special case of term equivalent symmetries, showing that such symmetries can be found in low-degree polynomial time. We show how to break an exponential number of these symmetries with added constraints whose number is linear in the size of the domain. We demonstrate the effectiveness of these techniques through experiments in two relational domains. We also discuss the connections between relational symmetry breaking and work on lifted inference in statistical-relational reasoning.

SUMMARY        The work concerns symmetries in relational models, which are       permutations of the set of atoms (i.e. predicates applied to       constants) that do not change a relational theory.A model of       a theory is a truth-assignment for each atom that makes the       theory true.A symmetry of a theory can therefore permute the       atoms in a model without changing whether the model satisfies       the theory.The authors define term symmetries to be those       symmetries that can be expressed as just a permutation of the       constants appearing in the atoms, rather than of the atoms       themselves.They further define term-equivalent symmetries as       term symmetries that can be expressed as a partition of the       constants, with the symmetries being all permutations that map       each constant to another in the same block.        Such symmetries thus define an equivalence relation on models,       with equivalence being with regard to satisfiability, weight,       or another objective.The idea of symmetry-breaking is to       introduce constraints to the theory that eliminate some       otherwise valid models, while making sure that at least one       model in each equivalence class survives.If implemented as a       pre-processing step to an inference or satisfiability       algorithm, this would serve to reduce the search space.        QUALITY        At a first glance, the work appears to be of fairly high       quality.The definitions are clear, and necessary theorems       are carefully stated and proved.        CLARITY        Having some knowledge of the field but not a deep background,       I found the exposition rather hard to follow.While the basic       definitions and concepts are clear enough (if rather dense), a       running example domain would have been helpful to illustrate       the types of symmetries being considered.        Some of the results seem counter-intuitive, but this is not       discussed.For example, the class of term symmetries being       larger than term-equivalent, one would expect that breaking       the former would result in a bigger speed-up than just       breaking the latter.The opposite seems to be the case in       many experiments; the "Term" procedure sometimes takes longer.       Some clarification would be appreciated on whether this is       because the term symmetry-breaking propositions cause a       slow-down.        ORIGINALITY AND SIGNIFICANCE        While the paper presents original algorithms for detecting and       breaking term and term-equivalent symmetries, the significance       of these symmetry classes was not immediately clear.An       argument should be made that these classes are sufficiently       rich and expressive to simplify real-world problems.        MINOR CORRECTIONS        In lines 188 and 189, "uniquely color" should be "unique       color". In line 88, "unique type" should perhaps be "single       type", since multiple constants can have the same type.