A statistical model for tensor PCA
We consider the Principal Component Analysis problem for large tensors of arbitrary order k under a single-spike (or rank-one plus noise) model. On the one hand, we use information theory, and recent results in probability theory, to establish necessary and sufficient conditions under which the principal component can be estimated using unbounded computational resources. It turns out that this is possible as soon as the signal-to-noise ratio β becomes larger than C √ k log k (and in particular β can remain bounded as the problem dimensions increase). On the other hand, we analyze several polynomial-time estimation algorithms, based on tensor unfolding, power iteration and message passing ideas from graphical models. We show that, unless the signal-to-noise ratio diverges in the system dimensions, none of these approaches succeeds. This is possibly related to a fundamental limitation of computationally tractable estimators for this problem. We discuss various initializations for tensor power iteration, and show that a tractable initialization based on the spectrum of the unfolded tensor outperforms significantly baseline methods, statistically and computationally. Finally, we consider the case in which additional side information is available about the unknown signal. We characterize the amount of side information that allows the iterative algorithms to converge to a good estimate.

Description:The paper presents a theoretical analysis of exact and approximate MLestimation of a simple form of tensor factorization. Namely, we aregiven a tensor X (of rank k and dimension n) which equals the k-foldtensor product of some vector v_0 with itself, plus a noise tensorwith components being iid Gaussian. The task is to estimate v_0 fromX. The task leads to a non-convex optimization problem overv_0.Assuming non-zero noise, the task is NP hard. The paper assumesseveral known algorithms to solve the task approximately:matricization (reshaping X to a matrix and solving matrixfactorization), power iterations, message passing, and message passinginitialized by a guiding information which is the true value of v_0plus noise.The following is proved:1) The difference between the exact ML estimate (requiring exponentialtime) of v_0 and the true v_0 is small with high probability if SNR isgreater than a threshold, which is a polynomial function of k. Thismeans, the exact ML estimation is succesfull for a reasonable SNR thatscales well with tensor rank.2) All approximate ML estimates with no guiding information (i.e.,matricization, power iterations, and unguided message passing) aresucessfull only for SNR larger than an exponential function of k. Thatis, for larger k these methods can yield very bad estimates.3) The guiding information can make approximate message passingsucessfull, provided that it the SNR of this guiding information isreasonably low.These results are demonstrated on simple experiments on synthetic dataand k=3.Comments:The achieved results are valuable and non-trivial and I recommend toaccept the paper. However, I do have a number of objections.My most important objection: The paper presents a *statistical* analysis of exact and approximate ML estimation of tensor factorization. However, it would be helpful tofirst summarize known results on hardness of the ML estimation problemfrom computational complexity point of view, without anystatistics. How hard is to *approximate* the Tensor PCAproblem (on lines 59-60)? You only say that it is NP-hard, but is it in APX, does it have a PTAS? How do these results relate to your statistical analysis?The authors informally conjecture (see abstract) that the founddichotomy between the exact (intractable) and approximate (tractable)ML estimation can be valid in general: all tractable algorithms for MLestimation are inevitably unsuccesfull. It would be nice to see morearguments for this conjecture. E.g., do you believe that all existing tractable approximate algorithms fall more or less to one of these classes?In Theorem 3, you assume that the result of the matricizationalgorithm is the top singular vector w. But this is not really thedesired result because w has yet to be (approximately) decomposed tothe (\lceil k/2\rceil)-fold tensor product of vector v_0 withitself. Can this affect your result?In experiments, you may wish to compute also the exact ML estimate,which I believe is tractable for fixed k=3. This would empiricallysupport the results from Section 2.In experiments, considering only k=3 may hide some important insights (how does the accuracy scale with k?).Clarity of the paper should be improved. The authors do not help the reader to follow this mathematically very difficult paper. Some parts were apparentlywritten in haste. There is a number of typos, e.g.:Theorem 1: C_1 in (7) should be C_2Lemma 2: Model modelReference [18] is incomplete.Many English typos in Section 5 and in the supplement.Finally, let me admit that the topic is not exactly my expertise andtherefore verifying the proofs in detail is beyond my abilities.  Themain paper is only a summary of results and all the proofs are in thesupplement. This material is mathematically very heavy and I believeonly real specialists can understand it in detail.


Stochastic Multi-Armed-Bandit Problem with Non-stationary Rewards
In a multi-armed bandit (MAB) problem a gambler needs to choose at each round of play one of K arms, each characterized by an unknown reward distribution. Reward realizations are only observed when an arm is selected, and the gambler's objective is to maximize his cumulative expected earnings over some given horizon of play T . To do this, the gambler needs to acquire information about arms (exploration) while simultaneously optimizing immediate rewards (exploitation); the price paid due to this trade off is often referred to as the regret, and the main question is how small can this price be as a function of the horizon length T . This problem has been studied extensively when the reward distributions do not change over time; an assumption that supports a sharp characterization of the regret, yet is often violated in practical settings. In this paper, we focus on a MAB formulation which allows for a broad range of temporal uncertainties in the rewards, while still maintaining mathematical tractability. We fully characterize the (regret) complexity of this class of MAB problems by establishing a direct link between the extent of allowable reward "variation" and the minimal achievable regret, and by establishing a connection between the adversarial and the stochastic MAB frameworks.

Secondary Review: The authors have replied to my concerns in a satisfactory manner. In particular, the problem with proof of Thm 1 can indeed be mitigated by the choice of epsilon suggested by the authors, although the second last inequality in their proposed derivation seems strange (the additional V_t^{2/3} would appear unnecessary) and, after noting that the "first part still holds" they can move directly to the case when \epsilon = 1/4 \sqrt{K/\tilde{\Delta}_T}. About line 424, their meaning is clear, however I see no reason to exclude the T^{1/3} term, as it provides information about the initial performance, and leaving it out all together would be misleading.Thanks to the authors for correcting their derivations, and I am happy to change my review scores accordingly.Initial Review:Summary of paper: The authors consider the multi-armed bandit (MAB) problem in a setting where the rewards are generated by non-stationary distributions. In particular they introduce a quantity ($V_T$ in the paper) which upper bounds the amount that the means of the reward distributions are allowed to vary by. They consider classes of MAB problems defined through admissible ranges for $V_T$, and they measure the worst-case regret of algorithms against a (non-stationary) oracle over these classes. They give a result lower bounding this worst-case regret of any strategy for these problems in terms of $V_T$, the time horizon, and the number of arms, and they give a matching upper bound on an epoch-variant of the Exp3 strategy. Thus they show a tight dependency of this class of problems on the quantity $V_T$, and motivate its interpretation as a quantitative measure of the cost of introducing non-stationarity into the standard MAB problem.Quality and Clarity: The paper is well written in parts, but not in others. In particular I found it hard to follow the details of the calculations in the proofs, and the many new notations. First two major issues:- It seems to me that there is a mistake in the proof of Theorem 1. Specifically on line 119 in the appendix I do not understand how the second inequality follows from the assumption that $T\ge KV_T$. Indeed, it seems to me that the numerator on the LHS of this inequality is greater than or equal to $T^{4/3}/8$ iff $(KV_T)/T \ge (1+\sqrt{\log(4/3)}/2)^3/2$, which is not implied by the assumption mentioned. I would appreciate it if the authors could clarify how the argument proceeds at this point.- I have difficulty in reconstruction the bound (a) on line 424 in the proof of Theorem 2. In particular, using the authors assumption that $V_T\ge K^{-1}$ I can bound the second term on the RHS of the inequality above (a) by $K(\log K)^{2/3}T^{1/3}$. Crucially, here there is a linear dependence on K. I would appreciate it if the authors could clarify how this becomes a dependence on $(KV_T)^{1/3}$. This last derivation is of great importance, as if it does not hold, then the bound is not tight with the lower bound provided in Theorem 1.I list also a few further suggestions that would have helped me to read the paper more fluently:- Some notation and notation and nomenclature differed from my experience of related literature, whereas it could be easily brought in line with it. In particular I found the "Admissible policies, performance and regret" paragraph did not introduce policies in a standard way, as they are introduced in many bandit and MDP papers (I think it is more normal to treat \pi as a distribution over the arm set that is measurable with respect to the history, and to denote by a_t, or I_t the chosen arm at time t).- The two paragraphs after Theorem 2 could be emphasised more since they are important and interesting consequences of the theorem and its proof.- I do not understand why the proof of Theorem 2 is in an Appendix within the main submission. It would be more natural for it to be a section before the discussion, or that it come immediately after Theorem 2.- The connection with reference [27] is often alluded to, but not in the end explained in any detail. It seems all of these allusions could therefore be in one place, for example the discussion section.Originality and significance: The contribution is original, and of significant interest to the NIPS community. It adds new understanding to the interface between stationary and non-stationary MAB problems. This should be of practical interest, although the authors do not directly suggest any real-world applications where the quantity $V_T$ can be interpreted.


(Almost) No Label No Cry
In Learning with Label Proportions (LLP), the objective is to learn a supervised classifier when, instead of labels, only label proportions for bags of observations are known. This setting has broad practical relevance, in particular for privacy preserving data processing. We first show that the mean operator, a statistic which aggregates all labels, is minimally sufficient for the minimization of many proper scoring losses with linear (or kernelized) classifiers without using labels. We provide a fast learning algorithm that estimates the mean operator via a manifold regularizer with guaranteed approximation bounds. Then, we present an iterative learning algorithm that uses this as initialization. We ground this algorithm in Rademacher-style generalization bounds that fit the LLP setting, introducing a generalization of Rademacher complexity and a Label Proportion Complexity measure. This latter algorithm optimizes tractable bounds for the corresponding bag-empirical risk. Experiments are provided on fourteen domains, whose size ranges up to ≈300K observations. They display that our algorithms are scalable and tend to consistently outperform the state of the art in LLP. Moreover, in many cases, our algorithms compete with or are just percents of AUC away from the Oracle that learns knowing all labels. On the largest domains, half a dozen proportions can suffice, i.e. roughly 40K times less than the total number of labels.

Summary: This paper presents new algorithms for the task of learning with label proportions (LLP). One of the proposed algorithm, the Laplacian Mean Map algorithm, extends previously proposed algorithm called Mean Map algorithm. It doesn’t pose homogeneity assumption but uses the Laplacian of the bag similarities. The Alternating Mean Map algorithm, an iterative variant of the LMM that uses LMM for initialization, is also investigated. The paper presents generalization bounds for LMM and AMM. Experimental results using different datasets and domains show that the proposed methods achieve the state-of-the-art performance in many cases, yet both of the algorithms scale to a large dataset.Quality: Detailed proof and thorough experiments clearly support the soundness and effectiveness of the proposed algorithms. Clarity: The paper is written very clearly. The supplemental material well complement the proofs and detailed experimental results.  Originality: Although application of manifold regularization and iterative procedure themselves is a natural extension to the Mean Map algorithm, it is worth publishing. Significance: Since the algorithms are easy to implement and scale to a large dataset, they will soon become strong baselines in LLP settings. Privacy-preserving technologies may also benefit from these algorithms.


A provable SVD-based algorithm for learning topics in dominant admixture corpus
Topic models, such as Latent Dirichlet Allocation (LDA), posit that documents are drawn from admixtures of distributions over words, known as topics. The inference problem of recovering topics from such a collection of documents drawn from admixtures, is NP-hard. Making a strong assumption called separability, [4] gave the first provable algorithm for inference. For the widely used LDA model, [6] gave a provable algorithm using clever tensor-methods. But [4,6] do not learn topic vectors with bounded l 1 error (a natural measure for probability vectors). Our aim is to develop a model which makes intuitive and empirically supported assumptions and to design an algorithm with natural, simple components such as SVD, which provably solves the inference problem for the model with bounded l 1 error. A topic in LDA and other models is essentially characterized by a group of co-occurring words. Motivated by this, we introduce topic specific Catchwords, a group of words which occur with strictly greater frequency in a topic than any other topic individually and are required to have high frequency together rather than individually. A major contribution of the paper is to show that under this more realistic assumption, which is empirically verified on real corpora, a singular value decomposition (SVD) based algorithm with a crucial pre-processing step of thresholding, can provably recover the topics from a collection of documents drawn from Dominant admixtures. Dominant admixtures are convex combination of distributions in which one distribution has a significantly higher contribution than the others. Apart from the simplicity of the algorithm, the sample complexity has near optimal dependence on w 0 , the lowest probability that a topic is dominant, and is better than [4]. Empirical evidence shows that on several real world corpora, both Catchwords and Dominant admixture assumptions hold and the proposed algorithm substantially outperforms the state of the art [5].

This paper describes an SVD-based method for provably recovering the word-topic distributions for collections of documents taken from "dominant admixtures".  This paper incrementally improves upon [5] by removing the assumption of topic "anchor words" (high probability in one topic and 0 probability in other topics) -- instead, there are weaker assumptions of topic "catchwords" (groups of words that occur strictly more frequently in one topic) and "dominant admixtures".  The assumption behind dominant admixtures is that there is a topic within each document that stands out among other topics.While I did not fully follow the proof of Theorem 3.1, I think the paper advances the line of research that solves LDA using SVD-type techniques.  The clarity of the paper could improve greatly by explaining the implications and important points of the theorems, assumptions, and lemmas.One critique I had was that this paper did seem to be incremental relative to [5] and the authors only performed empirical comparisons versus the Recover algorithm in [5].  Why not show the reconstruction errors and perplexity numbers for Gibbs sampling as well?  Most topic-modelers are more familiar with the traditional approximate inference schemes such as Gibbs sampling and variational inference, and it is important to compare to these well-known algorithms as well so that the community can get a sense of the performance of the proposed algorithm.I also did not fully understand the implications of the topic recovery evaluation.  It seems the original "ground-truth" M comes from LDA Gibbs sampling on the real data.  Isn't it expected a priori that the estimated M_TSVD would be closer to M than M_Recover because M_Recover is more sparse due to the Anchor word assumption?  In other words, if the data-generating distribution M were modified to be highly sparse, perhaps M_Recover would be produce lower reconstruction errors.  Also, the perplexity results on real data are not consistently better than Recover, so what is the evidence that this new approach has better performance?Other comments/questions:1) The generative process of dominant admixtures was not clearly spelled out in the Notation section.2) Please clarify that A_ij is fractional.   Line 085 says that it is a frequency.3) Did you try the approach on a data set where most of the documents did not have a dominant topic?  One way to create such a data set is to just fuse multiple documents (of completely different topics) into a super-document.  How would the approach perform in this regime?4) In Table 2, the NIPS results have lower error than the other data sets. Why not use a mean document length of 1000 for each of these semi-synthetic corpora to see if the performance can be replicated for all of these data set?5) Cited paper [7] also removes the Anchor word assumption, but there was no comparison made between that work and the author's work.


Combinatorial Pure Exploration of Multi-Armed Bandits
We study the combinatorial pure exploration (CPE) problem in the stochastic multi-armed bandit setting, where a learner explores a set of arms with the objective of identifying the optimal member of a decision class, which is a collection of subsets of arms with certain combinatorial structures such as size-K subsets, matchings, spanning trees or paths, etc. The CPE problem represents a rich class of pure exploration tasks which covers not only many existing models but also novel cases where the object of interest has a nontrivial combinatorial structure. In this paper, we provide a series of results for the general CPE problem. We present general learning algorithms which work for all decision classes that admit offline maximization oracles in both fixed confidence and fixed budget settings. We prove problem-dependent upper bounds of our algorithms. Our analysis exploits the combinatorial structures of the decision classes and introduces a new analytic tool. We also establish a general problem-dependent lower bound for the CPE problem. Our results show that the proposed algorithms achieve the optimal sample complexity (within logarithmic factors) for many decision classes. In addition, applying our results back to the problems of top-K arms identification and multiple bandit best arms identification, we recover the best available upper bounds up to constant factors and partially resolve a conjecture on the lower bounds.

This paper is (to the best of my knowledge) the first paper that studies best 'configuration' identification (or pure-exploration) in combinatorial multi-armed bandit. Two algorithms are proposed: CLUCB in the fixed-confidence setting and CSAR in the fixed-budget setting, and a lower bound is given in the fixed-confidence setting. The algorithms generalize is a very nice and natural way the LUCB and SAR algorithm designed for K best arms (or TopK) identification. As highlighted by the authors, the main strenght of the algorithms is that they treat in a unified way every combinatorial bandit problem. Their analysis feature a complexity term H can can be particularized to each combinatorial problem and appears natural. However, the paper suffer from two shortcomings: first, there is a small error in the proof of Theorem 1 (that can be corrected, an union bound is missing) and second, it is far too long. A NIPS reviewer will clearly NOT read 40 pages... I would advise you to remove all extensions (Appendix B. Appendix C.2 and Appendix E.) and keep them for an extended paper in a journal (where they will be more likely to be really reviewed). Moreover, the references to the theory of matroids (already in the main text) are a bit difficult to get for a non-initiated reader (like myself), so you should either define matroids in the main text or not mention them. Theorem 1 is a high probabability bound on the number of samples from the arms needed to identify the best configuration in the fixed-confidence setting. Its proof consists in showing that on the event \xi on which for all t, the true means of all arms belong to their confidence interval, the algorithm outputs the right configuration and the number of sample used are upper bounded deterministically. And it is shown that the confidence intervals are chosen such that the event \xi has probability larger than 1 - \delta. This fact is actually not completely true with your choices of confidence intervals: in the display at line 706, the number of draws of arm e T_t(e) is random, so Hoeffding inequality cannot be applied directly: you should add a union bound (summing over all the possible values for T_t from 1 to t), which would lead to a log(CKt^3/\delta) -with some constant C- in your confidence interval. On the event \xi, the upper bound on the number of samples strongly relies on Lemma 10, which resorts to the width of a class \cM and to the concept of exchange class, which is particular to combinatorial problems. Finally, in the end of the proof of Theorem 1, you say that you get the upper bound "solving for T". Would it be possible that you really give an explicit upper bound on T rather than a big O?I did not read the proof of the analysis of CSAR, but I guess it uses similar tools on exchange classes pluged into the analysis of SAR (and I hope other reviewers will confirm it is right). The quality of the paper is good, it is well-structured, except for the presence of "Related works" in Section 6, which I think would be more naturally presented in the introduction. There are some weird formulations and english mistakes at some places, some of them are: l.24 "we prove problem-dependent upper bounds of our algorithm" -> for our algorithmsl.102 " a probability of error bound of the CSAR algorithm" -> an upper bound on the probability of error of the CSAR algorithml.113 "standard assumptions of the stochastic MAB" -> in the stochastic MABsecond paragraph of page 3 : you use too much "the learner" (he or him could be fine at some places)paragraph 3 of the same page: "two different stopping conditions", I would rather say "constraints" (since in the fixed-budget setting there is no stopping condition)l.137 -> we present THE CLUCB algorithml.187 -> of -> forIt is a bit difficult from the main text to really have an intuition on what the width of an exchange class represents, but I don't really know how to improve that.   There is though a small error in the proof that must be corrected


QUIC & DIRTY: A Quadratic Approximation Approach for Dirty Statistical Models
In this paper, we develop a family of algorithms for optimizing "superpositionstructured" or "dirty" statistical estimators for high-dimensional problems involving the minimization of the sum of a smooth loss function with a hybrid regularization. Most of the current approaches are first-order methods, including proximal gradient or Alternating Direction Method of Multipliers (ADMM). We propose a new family of second-order methods where we approximate the loss function using quadratic approximation. The superposition structured regularizer then leads to a subproblem that can be efficiently solved by alternating minimization. We propose a general active subspace selection approach to speed up the solver by utilizing the low-dimensional structure given by the regularizers, and provide convergence guarantees for our algorithm. Empirically, we show that our approach is more than 10 times faster than state-of-the-art first-order approaches for the latent variable graphical model selection problems and multi-task learning problems when there is more than one regularizer. For these problems, our approach appears to be the first algorithm that can extend active subspace ideas to multiple regularizers.

The paper addresses the question of estimating so called dirty models in which the parameter is a sum of "simple parameters", complexities of which are individually regularized. The paper proposes a subspace selection strategy that makes quadratic approximation feasible. The theoretic development builds on the notion of decomposable norms. I have to confess that I have not followed this line of work very closely, so my comments are necessarily those of a generally knowledgeable person in machine learning and model selection.It appears that the work is a neat combination of ideas that have been published before, such as minimizing nuclear norm via active subspace selection (ICML 2014), and using quadratic approximation for sparse inverse covariance estimation (NIPS 2011), but the work is now carried out in a more general setting of decomposable norms. To me, this appears elegant, but some questions remain. The algorithmic achievement of 10-fold speedup does not sound very impressive, such factors are usually easily achieved by optimizing the implementation, thus the question becomes how easy it is to optimize (say parallelize etc.) the most demanding parts of the algorithm. For very practically oriented person the achievement may not appear great.The presentation of the theory is in general rather clear even if somewhat compact possibly due to page limit. The writing is not impeccable - some of the language errors are distracting to the level that hampers comprehension. The use of citations as a part of sentence is generally not encouraged and it creates new grammatical problems: should we write "[7, 5] consider the estimation ..." or "[7, 5] considers the estimation ..."? Sometimes poor punctuation creates problems like in "[14] in turn use a superposition ..." or "we consider instead a proximal ...". At worst the sentences are ungrammatical to the level of being incomprehensible like the sentence: "Overall, our algorithmic ...." on page 2. Writing formulas inside the text creates a set of problems of its own and sometimes the result is awkward - too often do we see a line ending in equal-sign (say 0 = \newline). On page 7, the formula breaches the margin as does the figure 1 and the table 1 on page 8.In general, it is customary to have some kind of conclusion or discussion in the paper. Now there is none. For example, discussion about the future directions would be welcome, since it is not clear to me where to go from here.In the algorithm 1, line 7, the superscript (t) should probably be (r), and updating the sum on line 8 looks strange (one should update variables rather than expressions).


Online combinatorial optimization with stochastic decision sets and adversarial losses
Most work on sequential learning assumes a fixed set of actions that are available all the time. However, in practice, actions can consist of picking subsets of readings from sensors that may break from time to time, road segments that can be blocked or goods that are out of stock. In this paper we study learning algorithms that are able to deal with stochastic availability of such unreliable composite actions. We propose and analyze algorithms based on the Follow-The-PerturbedLeader prediction method for several learning settings differing in the feedback provided to the learner. Our algorithms rely on a novel loss estimation technique that we call Counting Asleep Times. We deliver regret bounds for our algorithms for the previously studied full information and (semi-)bandit settings, as well as a natural middle point between the two that we call the restricted information setting. A special consequence of our results is a significant improvement of the best known performance guarantees achieved by an efficient algorithm for the sleeping bandit problem with stochastic availability. Finally, we evaluate our algorithms empirically and show their improvement over the known approaches.

The paper is clear and well written. The paper proposes an algorithm for a problem close to the sleeping experts. As in the case of sleeping experts, the decision set is available stochastically and the rewards are chosen by a non oblivious adversary. Here, the goal of the learner is to sequentially select its actions from a combinatorial decision set. The sleeping expert problem is important in practice, and the combinatorial optimization corresponds to the case where the decision set is structured by a list of actions, for instance in the case of advertising, the list of ads to display for a profile. As in the case of sleeping experts, the proposed algorithm is based on the well-known Follow-the-Perturbed-Leader algorithm. It is analyzed for three different feedbacks: full information, restricted information (the value of the loss function is revealed for available actions), and semi-bandit (the feedback is revealed for chosen actions).  As in the case of sleeping expert, the regret bounds are given with respect to the best fixed policy.The originality of the paper lies in the way in which the availability of decisions is taken into account. The authors propose a smart and efficient method to obtain unbiased loss estimation when the availability distribution is unknown: when an action becomes unavailable, its loss is estimated by its last observation. The reason of this simple approach is well explained and analyzed.  This loss estimation allows improving experimental results of the previous approach BSFPL (sleeping experts) since there is no exploration time to estimate the availability of actions.However, except the availability of decisions, the algorithms and the mathematical framework come from “An Efficient Algorithm for Learning with Semi-Bandit Feedback”, which is given as reference. To sum up, this work is an adaptation of algorithms proposed in the above paper to the sleeping expert problem.Furthermore, the paper is not self-sufficient, and to reproduce the analysis, the reader has to extract some statements in the demonstrations of the above paper, which are used here. I suggest that the authors provide all the demonstrations in the extended version. To improve the paper, I also suggest that the distinction between the decision vector and the action is done: the learner chooses a decision vector $V_t$, where each dimension $V_{i,t}$ is an action.There are some typos: line 211, and line 232.


Expectation-Maximization for Learning Determinantal Point Processes
A determinantal point process (DPP) is a probabilistic model of set diversity compactly parameterized by a positive semi-definite kernel matrix. To fit a DPP to a given task, we would like to learn the entries of its kernel matrix by maximizing the log-likelihood of the available data. However, log-likelihood is non-convex in the entries of the kernel matrix, and this learning problem is conjectured to be NP-hard [1]. Thus, previous work has instead focused on more restricted convex learning settings: learning only a single weight for each row of the kernel matrix [2], or learning weights for a linear combination of DPPs with fixed kernel matrices [3]. In this work we propose a novel algorithm for learning the full kernel matrix. By changing the kernel parameterization from matrix entries to eigenvalues and eigenvectors, and then lower-bounding the likelihood in the manner of expectation-maximization algorithms, we obtain an effective optimization procedure. We test our method on a real-world product recommendation task, and achieve relative gains of up to 16.5% in test log-likelihood compared to the naive approach of maximizing likelihood by projected gradient ascent on the entries of the kernel matrix.

The paper presents and EM algorithm for learning Determinantal Point Processes. The method combines several nontrivial ingredients: eigendecomposition of the kernel, optimisation on the Stiefel manifold, representation of DPPs as mixtures of elementary DPPs, and lower-bounding the log-likelihood by Jensen 's Inequality.  The experiments show favourable behaviour of the proposed approach compared to a simpler "K-Ascent" scheme. Quality: As far as I can tell the method is well-founded, and is believable that the methods should work as stated. However,the evaluation of the method could be improved. The experiments are quite limited and do not compare against alternative approaches for subset selection (e.g. those mentioned in the introduction), so it is not clear how much we benefit from the DPP approach.Clarity: The paper is not particularly clearly written. write-up is somewhat sloppy and hard to follow. Not all notation is defined and sometimes definition comes after using it. The writing is also unnecessarily convoluted at times, mostly because the text is structured to describe a process how one ends up with the finally proposed approach, with side-remarks of possible alternative approaches and problems interleaved with the main thread. I would have preferred to present the developed method as clearly as possible and postpone discussion to a later stage.Originality: For me the paper looks quite original, I am not aware of similar work.Significance: I think the method derived in the paper presents a solid advance and a useful contribution to subset selection literature.Details:- p.2. line 81 the marginal kernel K is not defined before use. What kind of kernel qualifies as marginal kernel?- p.2. l 100: V is not defined- p.3. l 159: "minimises the change to Frobenius norm" is unclearly stated. It would be better to say "chooses the closest (in Frobenius norm) PSD matrix to Q".- p 4. I found this section hard to understand. Rather than referring to [26] it would be nice to outline your (simpler) approach directly so that the reader is not forced to lookup the more complex original method.  - p.4 line 189: what does "weight" refer to?- p. 7 line 324. I don't quite get the rationale in (30). Why is m_{ij} not used here? Why would simply using the empirical covariance matrix as the kernel not have worked as initialisation?


Extracting Certainty from Uncertainty: Transductive Pairwise Classification from Pairwise Similarities
In this work, we study the problem of transductive pairwise classification from pairwise similarities 1 . The goal of transductive pairwise classification from pairwise similarities is to infer the pairwise class relationships, to which we refer as pairwise labels, between all examples given a subset of class relationships for a small set of examples, to which we refer as labeled examples. We propose a very simple yet effective algorithm that consists of two simple steps: the first step is to complete the sub-matrix corresponding to the labeled examples and the second step is to reconstruct the label matrix from the completed sub-matrix and the provided similarity matrix. Our analysis exhibits that under several mild preconditions we can recover the label matrix with a small error, if the top eigen-space that corresponds to the largest eigenvalues of the similarity matrix covers well the column space of label matrix and is subject to a low coherence, and the number of observed pairwise labels is sufficiently enough. We demonstrate the effectiveness of the proposed algorithm by several experiments.

The authors study problem of transductive pairwise classification and propose a simple algorithm to address the task by using two step approach: 1) sub-matrix completion corresponding to the labeled examples 2) label matrix reconstruction from the provided similarity matrix and the completed sub-matrix. Furthermore, authors give theoretical result on reconstruction error and empirically evaluate performance of the proposed algorithm. On one hand, it is certainly useful and interesting to consider transductive pairwise classification as a matrix completion problem. On the other hand, a natural setting for such task could be a semi-supervised or transductive preference learning. A large body of research exists on (ss) preference learning and ranking (e.g. work of Nir Ailon, Chu Wei, Johannes Fürnkranz, Eyke Hüllermeier, and others). These algorithms are frequently applied for link prediction, protein sequence ranking, etc, - basically all of the examples authors use to motivate their method. Why is this "setting" missing from the manuscript? Short description/reference to the relevant pairwise classification/preference learning algorithms as well as empirical comparison (e.g. to semi-supervised gaussian processes for preference learning) would further strengthen the paper.The authors suggest notable differences of their approach (in particular transductive  setting) compared to the algorithms presented in [25] and [23], however, a clear motivation for the proposed algorithm is lacking. While theoretical contribution of the paper is impressive, a related analysis has been described in [25] for semi-supervised setting and the strategy for perfect matrix recovery is quite similar to the one presented by the authors.   The experimental results section could be improved. For example, a comparison of several baseline methods on a synthetic data would be better than reporting the TPCMC result only. Also, it is not clear to me why "comparable" methods that can use side information (as in [25]) are not included in the evaluation. In fact, comparison is done only with two clustering techniques. Finally, performance of the methods such as SC or SKL greatly depends on a number of parameters, choice of kernel, they way Laplacian matrix is construed, etc. How were these parameters estimated? Were the experiments performed multiple times and the average results reported as in [25]?  Unless some of these questions are addressed it is difficult to agree with the last statement in the manuscript, that is "empirical evaluations have verified the effectiveness of the proposed algorithm".


Fairness in Multi-Agent Sequential Decision-Making
We define a fairness solution criterion for multi-agent decision-making problems, where agents have local interests. This new criterion aims to maximize the worst performance of agents with a consideration on the overall performance. We develop a simple linear programming approach and a more scalable game-theoretic approach for computing an optimal fairness policy. This game-theoretic approach formulates this fairness optimization as a two-player zero-sum game and employs an iterative algorithm for finding a Nash equilibrium, corresponding to an optimal fairness policy. We scale up this approach by exploiting problem structure and value function approximation. Our experiments on resource allocation problems show that this fairness criterion provides a more favorable solution than the utilitarian criterion, and that our gametheoretic approach is significantly faster than linear programming.

This paper proposes a policy selection criterion (regularized maximin fairness) for multi-agent factored MDPs that is equivalent to the most utilitarian of egalitarian policies. The paper then presents an algorithm to find such a policy based on framing the problem as a zero sum problem between a utilitarian player and an egalitarian player. The algorithm operates over multiple iterations of solving standard utilitarian multi-agent factored MDPs.The paper is well written with nice theoretical proofs, and the idea of viewing the problem as a zero sum game is compelling. However, I believe this view has unnecessarily complicated the solution. Also, I am worried that the problem itself is not strongly motivated.- Since the problem being solved is ultimately centralized, the need for a different selection criteria must be due to a need for greater model expressiveness or conciseness. Its not clear to me that the problems couldn’t just be modeled in a different way (perhaps with additional states) using a standard global reward function that expresses the goal they wish to maximize (i.e. throughput). No argument is made that these problems don’t arise only due to lack of modeling skill. Even more importantly, utility theory (as used by MDPs) relies on the expected utility hypothesis (risk neutral), but it seems to me that preferences over policies given by regularized maximin fairness breaks this assumption by preferring (motivationally) to be risk adverse.For example, suppose we have a problem where two agents can choose either to A) receive (1, 1) or B) flip a fair coin and randomly receive (0, 4) or (4, 0). Each individual’s utility preference is to flip the coin, but the reward received by centralized maximin would have a value of 0 if the coin was chosen. Or course, this is not how the maximin criteria is defined, but is how it should be defined given the motivation. All of the examples given (assembly plan, telecommunications, traffic lights) would prefer the (1,1) solution, hence my concern with the motivation, and if the  regularized maximin fairness is the proper means of modeling these problems. Even if the problem is repeated (so expectation makes more sense) agents would still want to be risk adverse. - I believe the algorithm should be viewed as column generation to solve the zero-sum game. The variables are weights on deterministic joint-policies (i.e. p* along with some q* for which the policy maximizes) such that the resulting solution is a convex combination of these policies. The objective becomes the value of the minimum player. Column generation proceeds as per the best response problem in Line 4. Where q* = the convex combination of q*’s that make up the current basis. One immediate benefit of this view, is that line 3 needs not be solved each time, instead the new column can be added to the current basis and one (or more) of the old columns can be removed. - I don’t understand why line 4 of the algorithm 1 is considered a single-agent MDP. Doesn’t the agent need to optimize over all agent’s actions, which is only efficient because this is a factored multi-agent MDP? Calling this a single-agent MDP is misleading. - Pet peeve: Use of the word “obviously” to avoid explanation. This is basically argument by ridicule (“if you don’t understand this step, then you’re stupid”) and should be avoided. If you don’t want to explain a step, don’t explain it. No need to insult your reader.- The discussion on decentralized policies and Proposition 2 seems out of place in this paper. DecPOMDPs are quite a different beast, and its not clear how this work would help decentralized policies. Proposition 2 is a general statement for any optimal centralized policy and is already known.


DFacTo: Distributed Factorization of Tensors
We present a technique for significantly speeding up Alternating Least Squares (ALS) and Gradient Descent (GD), two widely used algorithms for tensor factorization. By exploiting properties of the Khatri-Rao product, we show how to efficiently address a computationally challenging sub-step of both algorithms. Our algorithm, DFacTo, only requires two sparse matrix-vector products and is easy to parallelize. DFacTo is not only scalable but also on average 4 to 10 times faster than competing algorithms on a variety of datasets. For instance, DFacTo only takes 480 seconds on 4 machines to perform one iteration of the ALS algorithm and 1,143 seconds to perform one iteration of the GD algorithm on a 6.5 million × 2.5 million × 1.5 million dimensional tensor with 1.2 billion non-zero entries.

By exploiting the properties of the Khatri-Rao product, this paper proposes a technique to speed up Alternating Least Squares (ALS) and Gradient Descent (GD) algorithms for tensor factorization. Furthermore, parallel algorithms are also proposed for scalability. Experiments on large-scale tensors are used for evaluation.The problem of distributed (parallel) tensor factorization is interesting and challenging. The proposed (centralized) technique for speeding up ASL and GD seems to be interesting. But the experiments can be improved to make the paper more convincing. The detailed comments are listed as follows:1. The title (or contribution) of this paper is to design DISTRIBUTED methods for tensor factorization. However, only one short section (Section 3.1), which contains only several sentences, is used to describe the distributed algorithms. More details about the distributed framework should be provided and studied. Actually, the distributed programming models will greatly affect the scalability of machine learning algorithms. More specifically, the authors only mention MapReduce in the paper. However, it is well known that MapReduce is not a good programming model for machine learning algorithms with many iterations. Unfortunately, the algorithms proposed in this paper need many iterations.2. In experiments, the authors only report the results about time per iteration. As mentioned above, time per iteration cannot actually reflect the efficiency of a distributed learning algorithm. The authors should report the whole time for several iterations untill convergence, which will also include the communication time and file reading/writing overhead. 3. In Section 3, the C and B are not defined although we can find them in the supplementary materials.4. The reference style should be consistent.


Mind the Nuisance: Gaussian Process Classification using Privileged Noise
The learning with privileged information setting has recently attracted a lot of attention within the machine learning community, as it allows the integration of additional knowledge into the training process of a classifier, even when this comes in the form of a data modality that is not available at test time. Here, we show that privileged information can naturally be treated as noise in the latent function of a Gaussian process classifier (GPC). That is, in contrast to the standard GPC setting, the latent function is not just a nuisance but a feature: it becomes a natural measure of confidence about the training data by modulating the slope of the GPC probit likelihood function. Extensive experiments on public datasets show that the proposed GPC method using privileged noise, called GPC+, improves over a standard GPC without privileged knowledge, and also over the current state-of-the-art SVM-based method, SVM+. Moreover, we show that advanced neural networks and deep learning methods can be compressed as privileged information.

The paper suggests a new way for incorporating privileged data to the GPC model. The privileged data is used as a way to estimate the noise level of each input location. Both the noise level g and class probability f are learned jointly using EP with a one dimensional quadrature estimation. In my view this is an elegant way of using GPC with privileged information. If I understand correctly, adding function g() acts as a way to use locations with similar privileged information to better estimate the “noise” in that location.Quality: The paper is theoretically sound as it extends the GPC framework using an additive GP model. Overall the paper shows a good theoretical background and compares the results of several examples using different datasets. I would also like for the authors to mention something about the runtime complexity of GPC+.Clarity : Overall, the paper is clearly written. There are a few locations where editing of very long run-on sentences can help. There is also a italic run-away at line 38. The authors should spend less space explaining known ideas such as GPC and EP and explain more about the meaning of g(), e.g., what exactly does it mean hard/simple point, intuition using a simple example. Another interesting question is will the introduction of g actually hurts the model (as seen in a few examples in the results). Another important point that felt a bit swept under the rug was the approximation of Eq. 14. Why does this approximation work, and how well? Even if there is no room in the manuscript this should be discussed in the supplemental.Originality: The paper is original in the way it uses privilege information as the “noise” in the GPC model. Significance The paper showed improvements in several problems using very different datasets. My concern is that the improvement in accuracy might not justify the big increase in complexity of using the GPC+ framework. However, the idea behind GPC+ is nice and can allow a new perspective on the problem.


Spectral Learning of Mixture of Hidden Markov Models
In this paper, we propose a learning approach for the Mixture of Hidden Markov Models (MHMM) based on the Method of Moments (MoM). Computational advantages of MoM make MHMM learning amenable for large data sets. It is not possible to directly learn an MHMM with existing learning approaches, mainly due to a permutation ambiguity in the estimation process. We show that it is possible to resolve this ambiguity using the spectral properties of a global transition matrix even in the presence of estimation noise. We demonstrate the validity of our approach on synthetic and real data.

Summary: The authors consider the problem of learning a mixture of Hidden Markov Models. The authors first suggest using a spectral learning algorithm to learn a set of parameters for a hidden Markov model, and then provide a method for resolving the permutation ambiguity in the transition matrix to recover it’s underlying block-diagonal structure.Major Comments:1. I found this paper to be very well written for the most part. The experimental results section could be fleshed out a bit. In particular the 2. The authors rely on the fact that a mixture of Hidden Markov Models can be expressed as a single HMM. One of the main reasons, if not the primary reason, for learning a mixture of HMMs as opposed to a single HMM directly is that the block diagonal structure of the mixture of HMMs is sparse. This sparseness may be imposed on problems that are deemed to be too large to learn with a full model, especially if the problem is large and the amount of training data is small. However, in this paper, the author’s algorithm relies on *first* accurately estimating the full HMM parameters, then reversing the permutation in order to find the block diagonal structure. For most real-world problems this seems completely untenable. Do the author’s have suggestions for when their algorithm might be appropriate and when it might not be? 3. Again, in practice, the recovered transition matrix is not likely to be sparse. Is there any way to guarantee that the transition matrix learned by the spectral algorithms suggested in this paper will be (close to) sparse? I suspect that there may be many HMM models with full transition matrices that are close to the sparse model with block diagonal structure that the authors are looking for. It is not at all obvious that the matrix that is learned via [1,2] will be sparse with very small amounts of noise.4. I am really worried that the experimental results do not accurately reflect the reality of noise in the transition matrix. For example, in Section 4.1, the authors apply their algorithm to a transition matrix that has been permuted by noise sampled from a Dirichlet distribution… but this is a paper about learning mixtures of HMM parameters, not just depermuting matrices. Shouldn’t the authors have sampled observation data from a mixture of HMMs and then learned all of the parameters back from the *observed* data? This would give a much more accurate idea of how noise in observations and small quantities of data are actually reflected in the learned transition matrix, and how hard it is to depermute the matrix in the presence of this noise.5. The experiment in Section 4.2 is starting to get at what I was hoping for in the previous section, but there is nowhere near enough information to evaluate the quality of the experiment. In particular, I have no idea what the mixture of HMMs actually looked like. The actual parameters of the transition and observation matrix matter *a lot* for evaluating how easy or hard it is to learn the model from data. I would like to know more about the specifics of these parameters (the authors could put it in an appendix if they feel that space is at a premium).6. In table 1, why is EM initialized with spectral learning ever doing worse than spectral learning alone? Shouldn’t EM only improve the results?Minor Comments:1. I believe that the subscript of \Lambda in the bottom right corner of the first matrix in the proof in Section 3.2.1 should be K, not 2.2. Text and numbers are *way* too small in the figures.


Advances in Learning Bayesian Networks of Bounded Treewidth
This work presents novel algorithms for learning Bayesian networks of bounded treewidth. Both exact and approximate methods are developed. The exact method combines mixed integer linear programming formulations for structure learning and treewidth computation. The approximate method consists in sampling k-trees (maximal graphs of treewidth k), and subsequently selecting, exactly or approximately, the best structure whose moral graph is a subgraph of that k-tree. The approaches are empirically compared to each other and to state-of-the-art methods on a collection of public data sets with up to 100 variables.

This paper presents two very natural methods for searching for Bayesian networks of treewidth <= k.  The exact method uses a MILP to search jointly over (Bayesian network, elimination order, chordal graph induced by elimination on the moralized network).  The approximate method repeatedly samples a random k-tree and then uses dynamic programming or sampling to find the best network whose moralization is a subgraph of the k-tree (any network of treewidth <= k can be obtained in this way).The paper is clear and the methods seem like textbook material.  Unfortunately, I haven't had time to go read the closely related prior work cited at lines 051-058 or 068-071, so I can't comment on the size of the incremental contribution.  If none of the reviewers are familiar with those papers, then I think one of us should go look at them.Algorithm 1 (lines 236-244) does independent sampling of k-trees.  Would search by hill-climbing or simulated-annealing be more efficient, or a genetic algorithm?  It seems that such algorithms could be built in a similar way.  It is not obvious what distribution is sampled from by Algorithm 2 (lines 292-300).  Line 073 says it is uniform over the set of partial orders consistent with the k-tree, but this is not obvious.  In particular, the input clique tree is assumed to be rooted (i.e., the choice of root clique is NOT sampled), so doesn't that mean that some orders are impossible?  In general, I wish some effort were made to use the local score functions to influence the sampling distributions in the methods of section 4.  Presumably this could provide a speedup over uniform sampling.The experiments with the approximate methods could be presented more clearly.  All of these algorithms are anytime algorithms, so why not show the score (with error bars) as a function of time?   Other than that, experimental results and discussion generally look ok.  I notice one anomaly: why does the old method TWILP-10m strongly beat the new method MILP-10m on the wdbc dataset with treewidth >= 10?FURTHER COMMENTS TO AUTHORSline 220: You take a (k+1)-clique as the base case of a k-tree, but [7] takes a k-clique as the base case.  (I may prefer your definition but I note the discrepancy.)line 083: G = {N,A} should better be G = (N,A) or more standardly G = (V,E).lines 160-173: Please provide brief interpretations of the variables and constraints.  I believe that the p_{it} variables define the Bayesian network with the v_i specifying a variable ordering that is respected by the edge direction.  The y_{ij} variables specify a triangulated moralized version of this network (plus optional extra edges).  More precisely, the triangulation is obtained by at least one elimination ordering, and the direction of the y_{ij} edges respect this ordering, with the z_i certifying that the y_{ij} edges form a DAG.  You should gloss the constraints with comments to help the reader: (3a) ensures that the induced width is at most w, (3b) says "if y_{ij}=1  then z_j >= z_i + 1", (3c) says "y_{ij} ^ y_{ik} -> y_{jk} v y_{kj}" which creates an induced edge between j and k when eliminating i, (4a) enforces that each variable chooses exactly one of its allowed parent sets, and so on, with (4d) enforcing moralization.line 178: "the partial order induced by z_i": no, I think the partial order should be regarded as the transitive closure of the y_{ij}.  That is what determines the set of possible elimination orders that are consistent with this solution, i.e., the possible orders are the topological sorts of the DAG formed by the y_{ij}.  The z_i serve to certify that the y_{ij} do form a DAG, but they may commit to a stronger ordering than necessary, since two vertices that are unordered according to the y graph will not necessarily tie in their z values. (The z values are weakly ordered (i.e., totally ordered except for ties), but the particular choice of weak ordering is not determined by the y graph and is unimportant.)line 215: (ii) should be "second".  lines 228, 231: typos for "Dandelion."  Also, from [17] (section 4), it seems these should actually be called "generalized Dandelion codes," based on earlier work called Dandelion codes.line 226: please clarify whether these are k-trees OVER A GIVEN SET OF VERTICES, which is what you need for your application, in which the vertices denote given variables.  The definition of k-trees in the previous paragraph adds new vertices as needed, and one could think that "sampling a k-tree" means sampling from the space of such graphs up to isomorphism.Why does section 3 use w for the treewidth while section 4 uses k for the treewidth?line 249: you say O(n); what is n?  line 252: please give a parametric big-O complexity.  E.g., you say exponential but you don't give a formula . So the reader has to think too hard to figure out what the base of the exponential is, whether this exponential is multiplied by n or added to n, etc.  You do give something later on at line 258, but shouldn't the size of the parent sets (see line 154) figure into the formula?line 339: clarify "error estimates."  Does 50% that the current upper bound on model score was 1.5 times the model score of the current best solution?


Neurons as Monte Carlo Samplers: Bayesian Inference and Learning in Spiking Networks
We propose a spiking network model capable of performing both approximate inference and learning for any hidden Markov model. The lower layer sensory neurons detect noisy measurements of hidden world states. The higher layer neurons with recurrent connections infer a posterior distribution over world states from spike trains generated by sensory neurons. We show how such a neuronal network with synaptic plasticity can implement a form of Bayesian inference similar to Monte Carlo methods such as particle filtering. Each spike in the population of inference neurons represents a sample of a particular hidden world state. The spiking activity across the neural population approximates the posterior distribution of hidden state. The model provides a functional explanation for the Poissonlike noise commonly observed in cortical responses. Uncertainties in spike times provide the necessary variability for sampling during inference. Unlike previous models, the hidden world state is not observed by the sensory neurons, and the temporal dynamics of the hidden state is unknown. We demonstrate how such networks can sequentially learn hidden Markov models using a spike-timing dependent Hebbian learning rule and achieve power-law convergence rates.

1070: Neurons as Monte Carlo Samplers: Bayesian Inference and Learning in Spiking NetworksThe authors study a possible neuronal implementation of learning and inference in a hidden Markov model. As a major novelty, the authors propose that the stochasticity of synaptic transmission is directly involved in the implementation of stochasticity necessary for Monte Carlo sampling. The neurons used throughout the paper are binary threshold units and not spiking neurons. These binary neurons are able to provide useful insights into how a neuronal network may solve computational problems, but it is important to distinguish between implementations using binary units and spiking neurons. The authors include a short section about spike-based implementation in the appendix, but they do not demonstrate that the spike based implementation is able to perform the same tasks with similar performance. For example, the short term plasticity they propose in the supplementary material to prevent firing after multiple spikes from the recurrent or feed-forward inputs, is not going to help, as STP acts between spikes arriving from at a single synapse not on inputs arriving from a pool of neurons.Inference: The authors write in the introduction that previous work has been limited to binary word states. However, any discrete state space can be represented by a set of binary variables (just as they do in Fig 2b), hence its is not entirely clear how the paper differs from those previous studies in this aspect.From the derivation above Eq. 5. and the that C_W~L it follows that the release probability scales with 1/L. However, the release probability in cortical synapses is around 0.5. The authors need to comment on this difference. During the proof of Eq. 6 the authors take the limit that the spiking activity is sparse. However, it is not clear why this limit is relevant here, especially since sampling is more efficient in the high firing rate limit. The prediction for the Poisson variability also follows from the sparsity assumption and it should be confirmed by simulations.On Fig. 2a the authors should compare the approximate posterior inferred by the model with the true posterior and not only with the data. They should use a quantitative measure to test the performance of the network (e.g., KL-divergence). They show marginal An interesting prediction of the theory would be that the sum of the release probabilities at the outgoing synapses of a neuron is constant (C_W) while the same sum for the incoming synapses covaries with the marginal state probability.Learning:The learning rule: Eq. 11 and 12 should be replaced by Eq. 31 and 35 of the appendix. Learning is non-local, since it depends on the number of spikes in the (pre and postsynaptic) population (n) and not on the actual spiking of the neuron (s).  It is not clear from the data provided whether the learning rule converges to the right value in the case when both the emissions and the hidden dynamics are learned together.


Finding a sparse vector in a subspace: Linear sparsity using alternating directions
We consider the problem of recovering the sparsest vector in a subspace S ∈ R p with dim (S) = n. This problem can be considered a homogeneous variant of the sparse recovery problem, and finds applications in sparse dictionary learning, sparse PCA, and other problems in signal processing and machine learning. Simple convex heuristics for this problem provably break down when the fraction of nonzero entries in the target sparse vector substantially exceeds 1/ √ n. In contrast, we exhibit a relatively simple nonconvex approach based on alternating directions, which provably succeeds even when the fraction of nonzero entries is Ω(1). To our knowledge, this is the first practical algorithm to achieve this linear scaling. This result assumes a planted sparse model, in which the target sparse vector is embedded in an otherwise random subspace. Empirically, our proposed algorithm also succeeds in more challenging data models arising, e.g., from sparse dictionary learning.

The aim of the paper is to produce an algorithm and supporting analysis for the problem of finding a planted sparse vector in a random subspace. The paper motivates this problem as being a close relative of sparse recovery as well as having potential extensions to the dictionary learning model (in which more than one sparse vector is sought). Previous convex approaches to this problem undergo a phase transition when the number of nonzero elements in the sparse vector greatly exceed n^(-1/2). The paper finds an alternating directions based algorithm featuring a data-based initialization and an additional rounding step based on linear programming. This algorithm, in contrast to previous convex-based ones, succeeds (whp) with the target vector having a constant fraction of nonzero entries. The theoretical guarantee requires that the dimension p >= n^4 log^2 n, though empirically it seems to succeed for p linear in n. The paper produces experiments which demonstrate this linear phase transition on synthetic data, as well as showing a novel and "strange" experiment on real data demonstrating its potential role in discovering "unusual vectors" in a subspace.The paper is argued very clearly for the most part and is of high quality on its own terms. While the theoretical significance of this work is clear, the practical significance is not sufficiently clear. It is unclear to me whether the experiments on real data are of interest to practitioners. Furthermore, although the connection to dictionary learning is alluded to as a possible expansion for future works, this is not well argued for either empirically or theoretically. However, the authors have explained this connection in the rebuttal. Stronger arguments for practical importance will benefit the paper (though the authors have alluded to relations with other problems). I am not a direct expert in this particular area so I may not be able to fully appreciate its potential impact on various communities. I still find the paper very original, of high theoretical quality and even somewhat intriguing and I thus assigned the first (of two) impact scores. This was clearly the strongest paper I reviewed (among five).There are few typos that can be corrected: * "with with" in statement of Theorem 2.1 (it will also be good to define "with overwhelming probability")* Missing section reference (line 122)*(8) is not *the* Huber m-estimator over Yq (lines 137-8 are confusing).* Revise writing after first sentence of 192 (e.g., next sentence is not grammatically complete)I have read the authors' rebuttal and made very minor changes to my review. I congratulate the authors on their nice paper.


Recovery of Coherent Data via Low-Rank Dictionary Pursuit
The recently established RPCA [4] method provides a convenient way to restore low-rank matrices from grossly corrupted observations. While elegant in theory and powerful in reality, RPCA is not an ultimate solution to the low-rank matrix recovery problem. Indeed, its performance may not be perfect even when data are strictly low-rank. This is because RPCA ignores clustering structures of the data which are ubiquitous in applications. As the number of cluster grows, the coherence of data keeps increasing, and accordingly, the recovery performance of RPCA degrades. We show that the challenges raised by coherent data (i.e., data with high coherence) could be alleviated by Low-Rank Representation (LRR) [13], provided that the dictionary in LRR is configured appropriately. More precisely, we mathematically prove that if the dictionary itself is low-rank then LRR is immune to the coherence parameter which increases with the underlying cluster number. This provides an elementary principle for dealing with coherent data and naturally leads to a practical algorithm for obtaining proper dictionaries in unsupervised environments. Experiments on randomly generated matrices and real motion sequences verify our claims. See the full paper at arXiv:1404.4032.

The paper proposes to use Low Rank Representation (LRR) with some learned dictionary A to improve the effectiveness of RPCA. In particular, the authors show that the incoherence parameter \mu which often thought of as a bottleneck in the recovery guarantee in RPCA can in fact correspond to some additional clustering structure in the data. Under some condition on the dictionary A, the authors show that one can exploit such structure using LRR and partially remove the dependence on \mu and get a theoretically stronger guarantee in both exact low-rank and sparse decomposition and its noisy extension.The method is novel and significant to the field. It is closely related to the union-of-subspace structure assumed in previous subspace clustering papers such as “Wang and Xu: Noisy Sparse Subspace Clustering & Provable subspace clustering: When LRR meets SSC”, and “Soltanolkotabi and Candes: A geometric analysis of subspace clustering with outliers” “Soltanolkotabi et al: Robust subspace clustering” but are less explicit about the assumption on “clustering”, which is reasonable and more probably more general. The simulation and real data experiments verify the theoretical analysis and I think the new structure can be found in many other real applications too (such as text data as remarked in the paper).In summary, the paper contains a substantial contribution to the field of compressed sensing and I think it should be accepted by NIPS.That said, I do have a number of discussions and stylistic suggestions which are summarized in the detailed comments below. 1.	Line 80-82: Need some re-writing to make the description clearer. What do you mean by “our data”? Also even if the problem captures all the structures in the problem, there is still yet another condition to qualify to have “perfect recovery”.2.	Line 88: … is no longer a method of perfect (WHAT?).3.	Please proof-read the papers more carefully for problems like the above two. Be specific and precise about what to claim and what you are referring to in the text. In general, please work on clarity of the language in the paper.4.	Wang and Xu also have written in their paper “Noisy Sparse Subspace Clustering” how higher coherence parameter is actually good for subspace clustering (unlike in RPCA and matrix completion). The key difference from here is that the data need not be low-rank in overall, only every cluster has to be low-rank. The structure exploited here in this paper can be thought of as a combination of low-rank structure and union-of-subspace cluster structure.5.	It will be interesting to compare the proposed algorithm against first solving noisy subspace clustering with \ell_1 penalty then do PCA for each subspace. No provable guarantee exists to date for the later version though.6.	Section 4.2 essentially proposes a new method for solving subspace clustering problem with sparse corruption in the data. The method leads to significant improvements over the standard RPCA+SSC on Hopkins155 dataset which I think is a hidden contribution here in this paper and the authors should point it out.


A Bayesian model for identifying hierarchically organised states in neural population activity
Neural population activity in cortical circuits is not solely driven by external inputs, but is also modulated by endogenous states which vary on multiple time-scales. To understand information processing in cortical circuits, we need to understand the statistical structure of internal states and their interaction with sensory inputs. Here, we present a statistical model for extracting hierarchically organised neural population states from multi-channel recordings of neural spiking activity. Population states are modelled using a hidden Markov decision tree with state-dependent tuning parameters and a generalised linear observation model. We present a variational Bayesian inference algorithm for estimating the posterior distribution over parameters from neural population recordings. On simulated data, we show that we can identify the underlying sequence of population states and reconstruct the ground truth parameters. Using population recordings from visual cortex, we find that a model with two levels of population states outperforms both a one-state and a two-state generalised linear model. Finally, we find that modelling of state-dependence also improves the accuracy with which sensory stimuli can be decoded from the population response.

The firing properties of neurons in the mammalian brain are not constant over time. One kind of non-stationarity that has been observed is that under certain conditions, populations of neurons can switch between discrete states, across which their inputs and responsiveness can vary dramatically. In turn, there may be structure to this network state-space. Despite all this, much neurophysiological data has been analysed whilst ignoring these discrete spaces, or with hacky/ad-hoc approaches to separate out the states. This paper attempts to tackle such population datasets by integrating GLM-style modelling of stimulus-response relationships into a hierarchical discrete state-space framework. The authors construct a statistical model for neural responses under this framework, based on a hierarchical HMM (i.e., an HME), and approximate inference under it using VBEM. The authors then demonstrate its recovery of ground truth under a simulated dataset, and its application to real neural data.Quality: The statistical model and its inference appear clean and sound, and for the examples presented, it performs marginally (but significantly) better on both simulated and real data compared with less complex models (e.g. when priors are removed; or using a two-state rather than a hierarchical three state model). Having said that, while it is certainly an improvement over previously presented models within the same class (e.g. Escola et al (2011), Chen et al (2009)), the improvement in performance is pretty small. Moreover, there does not appear to be a strong argument that the *hierarchical* nature of this model actually adds anything. There is no explicit comparison with an equally powerful non-hierarchical model (e.g. 3-state HMMs with VB for the state model, to compare with the examples of 3-state HMEs). This leaves me scratching my head, since, according to the title, the hierarchical aspect of the state space is the novel component of this model. Without seeing the data, I worry a bit about whether this promise can be achieved on the real datasets in question, since the improvement of 3-state HMEs over 2-state HMMs is typically small (e.g. Fig 3C, Fig 4 caption). Of course, the authors can convince me otherwise by presenting the results, in which case I would give the paper score a bump. Otherwise, while I can appreciate the promise of this approach, the real data may not justify such a model. For instance, the "hierarchy of discrete states" may be better described by a continuous latent factor. I'm nevertheless in favour of acceptance since this work is sound and should be published, but its scope of utility may not yet have been realised. Clarity: The paper is clear and well-written. Some minor points:- line 51: "simple a global" -> "a simple global"- line 99: "as well as hierarchical relationships between trials". Not sure what this is meant to mean. The architecture of the HMDT is fixed (though its parameters are learned). What does "between trials" mean here?- line 106: abnormal sentence termination "non-geometrically".- Fig 1B: some captioning/labelling of the transition matrices is necessary -- I couldn't parse these- line 112: "ypically"- line 267: "where theta is the phase" -> should be phi- line 267: pairwise product -> outer product?- line 266-268: a little motivation for the chosen parameterisation of the stimulus would be useful (yes, it's tangential to the core of the paper, but it's a bit bewildering. These look like a filtering of the stimulus by a set of 15 direction-selective/non-selective and orientation-selective/non-selective simple cells, yes?)- line 321: demote Macaques to plain old macaques- Fig 4: not referenced in main paper- line 386: "for the characterized..."? - line 406: "both, better" omit comma- the British author and American author should wrestle it out to determine spelling conventionsOriginality: This is an improved statistical model above and beyond other recently published work. However, as mentioned above, I think a full comparison is necessary to fully evaluate this work.Significance: See comments above.


SerialRank: Spectral Ranking using Seriation
We describe a seriation algorithm for ranking a set of n items given pairwise comparisons between these items. Intuitively, the algorithm assigns similar rankings to items that compare similarly with all others. It does so by constructing a similarity matrix from pairwise comparisons, using seriation methods to reorder this matrix and construct a ranking. We first show that this spectral seriation algorithm recovers the true ranking when all pairwise comparisons are observed and consistent with a total order. We then show that ranking reconstruction is still exact even when some pairwise comparisons are corrupted or missing, and that seriation based spectral ranking is more robust to noise than other scoring methods. An additional benefit of the seriation formulation is that it allows us to solve semi-supervised ranking problems. Experiments on both synthetic and real datasets demonstrate that seriation based spectral ranking achieves competitive and in some cases superior performance compared to classical ranking methods.

Thanks for the replies to my questions. Of course, I do agree that my example is somewhat specific, although I still believe that it's meaningful with regard the general question regarding the information (not) represented by the similarity relation. I more or less agree on the other answers, although my doubts are not completely dispelled. In summary, I still think the paper puts forward an interesting idea and therefore encourage the authors to continue in this direction, even if the paper might not yet be ready for a publication on the level of the NIPS conference. Original review:The idea of reducing the problem of ranking to a problem of similarity analysis appears to be interesting, however, it is not sufficiently well justified in the paper. What exactly is the connection between the preference relation C and the similarity relation S? Do they represent the same information? If not, what information is added or removed, and why is this an advantage? In the case of a single pairwise preference a > b, represented by the relationC=  +1  +1 -1  +1 , one obtains a symmetric similarity relation, which means that the information about the direction of the preference is completely lost. Another example: If we have the chain of preferences a > b, b > c, c > d, i.e., the matrixc=   1     1     0     0    -1     1     1     0     0    -1     1     1     0     0    -1     1 , then (unless I made a mistake somewhere) the second-smallest eigenvalue of the Laplacian of S is not unique, and none of the eigenvectors corresponds to the ranking a > b > c > d one would expect to find. The reason could be grounded in the problem that the conversion of preferences into similarities does not capture implicit connections via transitivity. Instead, a non-observed preference contributes to the similarity between two items (by adding 1/2 according to (2)), just like a draw (which, in contrast to a non-observation, is indeed a confirmation of similarity). In this example, the similarity relation iss =    3.0000    2.0000    1.5000    2.0000    2.0000    3.5000    2.0000    1.5000    1.5000    2.0000    3.5000    2.0000    2.0000    1.5000    2.0000    3.0000As can be seen, a and d are quite similar, and in fact, a is more similar to d than to c (although it should be the other way around). This is because there is no comparison at all between these items, neither a direct one nor an indirect one via a third item. If we add the preference d > a to the above chain, we create a cycle, which makes it impossible to produce a reasonable ranking. Again, the second-smallest eigenvalue is not unique, but nevertheless, both Fiedler vectors suggest some ranking. Dies that make sense?The formal results in section 4 about robustness are technically interesting, however, they refer to very specific cases and compare to very specific alternative methods. For example, it's true that for a single pairwise inversion, the simple Borda count will produce ties. However, there are many other methods that can easily resolve the problem. Condition (7) is difficult to understand. The experiments are again interesting, however, I fail to see a clear advantage of the proposed method (for example, in comparison to BTL). By the way, in connection with the regularity assumptions specified in Definition 2.1 (R-matrix), you might be interested in recent work on ranking based on pairwise preferences induced by statistical rank models such as Mallows or Plackett-Luce. As has been shown, these models imply strong regularity of the pairwise preferences, and the C-matrix they induce is Toeplitz [1,2].References:[1] W. Cheng, E. Hüllermeier, W. Waegeman and V. Welker. Label Ranking with Partial Abstention based on Thresholded Probabilistic Models. NIPS-2012, Lake Tahoe, Nevada, USA.[2] R. Busa-Fekete, E. Hüllermeier, B. Szörenyi. Preference-based rank elicitation: The case of Mallows. ICML 2014.


Just-In-Time Learning for Fast and Flexible Inference
Much of research in machine learning has centered around the search for inference algorithms that are both general-purpose and efficient. The problem is extremely challenging and general inference remains computationally expensive. We seek to address this problem by observing that in most specific applications of a model, we typically only need to perform a small subset of all possible inference computations. Motivated by this, we introduce just-in-time learning, a framework for fast and flexible inference that learns to speed up inference at run-time. Through a series of experiments, we show how this framework can allow us to combine the flexibility of sampling with the efficiency of deterministic message-passing.

This paper is an extension of the recent Heess et al "Learning to Pass Expectation Propagation Messages". AIUI, Heess et al uses neural networks to learn an approximate mapping between input and output messages. In Hees et al, learning is done generically for each model, before specific problem instances are seen. Their approach has the drawback of requiring the compilation of synthetic training data, and being unable to adapt to model peculiarities during the message-passing phase. The present paper fixes both of these problems by learning input-output mappings as messages are being passed. It does this by falling back to an "oracle" for each message until the approximate mapping at that node has been suitably trained. The oracle is simply the (presumably expensive) way in which messages would have been computed in the absence of an approximation.The approach is tested on two synthetic problems, as well as on a probabilistic model of corn yield. Infer.NET is used throughout.The paper presents some important concepts, and I think it should be accepted because it is a "good enough" presentation of concepts which will have to be published at some point anyway. My main criticism is that the experiments seem biased in favor of the proposed method. For instance, the last subsection of Experiments says "We find that with around 100,000 samples the message for the yield factor can be computed accurately, making these by far the slowest computations in the inference procedure". The factor in Figure 5a looks pretty simple, I don't see why some sort of smooth approximation couldn't be used to compute messages going through it - 100,000 samples sounds excessive. At the same time, the proposed method for learning approximate message-operators based on random decision forests seems very slow. Perhaps another reviewer who is more familiar with expectation propagation and Infer.NET could comment on this.Perhaps also because I don't have a good background on expectation propagation, I find that after reading the paper I don't have a satisfactory intuition about why the proposed method, as well as the one in Heess et al, is needed. Do messages really jump around so much as they converge, that we need neural networks or random decision forests to speed this up? It would be nice if there was room to go into more detail about what's happening under the hood. Or am I missing something.Minor comments: "Regressor" should be defined, in Wikipedia it is defined as "independent variable", but I think the meaning is different in this paper. Expectation propagation (and Infer.NET?) should be mentioned in the abstract! "Related work" is usually Section 2, I am not sure why this paper puts it at Section 5.Clarity: The paper is fairly clear. Important details of experiments and models have been moved to Supplementary Material, which makes it a bit harder to understand what is happening in detail.Originality: The paper is original. It is to a certain degree an "engineering" paper, in that a number of different approximate techniques (EP, importance sampling, random forests) are drawn together without full justification. But I think this comes with the territory.I think I addressed quality and significance above.


Learning Optimal Commitment to Overcome Insecurity
Game-theoretic algorithms for physical security have made an impressive realworld impact. These algorithms compute an optimal strategy for the defender to commit to in a Stackelberg game, where the attacker observes the defender's strategy and best-responds. In order to build the game model, though, the payoffs of potential attackers for various outcomes must be estimated; inaccurate estimates can lead to significant inefficiencies. We design an algorithm that optimizes the defender's strategy with no prior information, by observing the attacker's responses to randomized deployments of resources and learning his priorities. In contrast to previous work, our algorithm requires a number of queries that is polynomial in the representation of the game.
security games is that even in the simplest security game, the number of pure defender strategies is exponential in the representation of the game. For example, if each of the defender's resources can protect one of two potential targets, there is an exponential number of ways in which resources can be assigned to targets. 1 Our approach and results. We design an algorithm that learns an (additively) -optimal strategy for the defender with probability 1 − δ, by asking a number of queries that is polynomial in the representation of the security game, and logarithmic in 1/ and 1/δ. Our algorithm is completely different from that of Letchford et al. [8]. Its novel ingredients include:
• We work in the space of feasible coverage probability vectors, i.e., we directly reason about the probability that each potential target is protected under a randomized defender strategy. Denoting the number of targets by n, this is an n-dimensional space. In contrast, Letchford et al. [8] study the exponential-dimensional space of randomized defender strategies. We observe that, in the space of feasible coverage probability vectors, the region associated with a specific best response for the attacker (i.e., a specific target being attacked) is convex.
• To optimize within each of these convex regions, we leverage techniques -developed by Tauman Kalai and Vempala [14] -for optimizing a linear objective function in an unknown convex region using only membership queries. In our setting, it is straightforward to build a membership oracle, but it is quite nontrivial to satisfy a key assumption of the foregoing result: that the optimization process starts from an interior point of the convex region. We do this by constructing a hierarchy of nested convex regions, and using smaller regions to obtain interior points in larger regions.
• We develop a method for efficiently discovering new regions. In contrast, Letchford et al. [8] find regions (in the high-dimensional space of randomized defender strategies) by sampling uniformly at random; their approach is inefficient when some regions are small.

This paper considers the problem of learning an attacker’s payoffs in a security game. In deployed application, game-theoretic solutions to security games have had much success but crucially rely upon experts to provide the payoffs for the attackers (terrorists, drug smugglers, etc) to the equilibrium solvers. This paper suggests replacing this expert-provided knowledge by learning the payoffs from the attacker by observing attacker responses in repeated plays of the attacker-defender game. The authors make use of techniques by Kalai and Vempala which uses an initial feasible point and a membership oracle to optimize the defender’s objective. To do so, the authors prove in this paper that the optimization region is reasonably well-behaved, most notably that there exists initial points that are sufficiently far from the boundaries of the optimization region. Using all of this, they show that the optimal defender strategy can be computed with a polynomial number of queries (plays of the security game), improving over the previous result of an exponential number of plays. On the high level, I think this paper has a good idea of attempting to correct for the uncertainty in payoff knowledge and uses sophisticated techniques in their chosen approach. If the best way to correct for this uncertainty was to explore the strategy space and find the exact optimal solution, then this paper does offer a significant improvement over the prior method to do so. Indeed, the techniques in this paper is non-trivial and has some nice applications of convex optimization methods to game theoretic settings, i.e. the interpretation of the attacker’s best response as the membership oracle. However, I’m not convinced that this approach is the right way of solving this problem and the authors don’t make an argument for it. Its not clear that a polynomial number of plays of this security game represents a reasonable amount of computation in the studied contexts. What is the rate that such games occur? Do attacker utility functions remain constant for long enough to actually learn in this context? I think this work could have benefited from empirical analysis in a couple of ways. First, I would have been very interested to see the practical convergence rate on some typical instances of these games (for example, instances of the games used for the empirical testing in [1]). Perhaps the convergence rate is practically very fast, so the concern about the number of games played is not necessary. Second, it would have been great to see the proposed method compared to some other strategies used to address this uncertainty like the robust optimization methods or distribution knowledge methods as mentioned by the authors in the introduction. Maybe its the case that the method in this paper finds such a better solution that its worthwhile to wait for the convergence of this algorithm. Finally, for realistic scenarios, is finding the exact optimal solution is the right objective, especially if the defender sacrifices a lot of interim utility in order to find it? A concept like regret might be better suited as it allows for the examination of the trade-offs between exploring the entire strategy-space versus exploiting good strategies that may be learned very quickly. [1]: Jain, Manish, et al. "Software assistants for randomized patrol planning for the lax airport police and the federal air marshal service." Interfaces 40.4 (2010): 267-290.


Learning Mixtures of Submodular Functions for Image Collection Summarization
We address the problem of image collection summarization by learning mixtures of submodular functions. Submodularity is useful for this problem since it naturally represents characteristics such as fidelity and diversity, desirable for any summary. Several previously proposed image summarization scoring methodologies, in fact, instinctively arrived at submodularity. We provide classes of submodular component functions (including some which are instantiated via a deep neural network) over which mixtures may be learnt. We formulate the learning of such mixtures as a supervised problem via large-margin structured prediction. As a loss function, and for automatic summary scoring, we introduce a novel summary evaluation method called V-ROUGE, and test both submodular and non-submodular optimization (using the submodular-supermodular procedure) to learn a mixture of submodular functions. Interestingly, using non-submodular optimization to learn submodular functions provides the best results. We also provide a new data set consisting of 14 real-world image collections along with many human-generated ground truth summaries collected using Amazon Mechanical Turk. We compare our method with previous work on this problem and show that our learning approach outperforms all competitors on this new data set. This paper provides, to our knowledge, the first systematic approach for quantifying the problem of image collection summarization, along with a new data set of image collections and human summaries.

This paper formulates the task of summarizing personal photo collections as submodular optimization, and proposes a max-margin formulation with multiple choices for loss functions to learn weights to these. A dataset of image collections with several human-generated summaries is presented, together with a scoring method, using which the proposed approach is evaluated. Experiments show that on this dataset, the method has a very good performance, even outperforming human labelers on average.QualityThis paper is a through investigation of the possible application of submodularity for image summarization. Extensive experiments were conducted to compare different optimization methods (suitable for different choices of loss functions). The authors also provide an ample amount of supplementary material, giving even more details.ClarityThe paper is well organized, the explanations are clear and also well-written in terms of English usage.A few remarks:- on line 144, does the term "compatible" have a special meaning, justifying the italics? (also, similar was the term "calibrate" on line 155.)- the relation between the components in section 6 of the supplementary material and the summary in section 5 of the paper could be clarified (or at least it could be mentioned that further information can be found in the supplementary material).OriginalityA major contribution of the paper is posing image set summarization as a submodular optimization problem; to the knowledge of this reviewer, this is a novel view of the problem. Together with a new dataset and the adaptation of ROUGE to a significantly different application domain, this paper has several novel contributions to the state of the art.SignificanceAccording to this reviewer, this work fits well in the topics of interest for NIPS, to which it makes a significant contribution.