Best-Arm Identification in Linear Bandits
We study the best-arm identification problem in linear bandit, where the rewards of the arms depend linearly on an unknown parameter θ * and the objective is to return the arm with the largest reward. We characterize the complexity of the problem and introduce sample allocation strategies that pull arms to identify the best arm with a fixed confidence, while minimizing the sample budget. In particular, we show the importance of exploiting the global linear structure to improve the estimate of the reward of near-optimal arms. We analyze the proposed strategies and compare their empirical performance. Finally, as a by-product of our analysis, we point out the connection to the G-optimality criterion used in optimal experimental design.

This paper is one of the first to study best arm identification in bandit models with correlated arms. The authors consider the fixed-confidence setting. A new complexity notion, that takes into account the correlation between arms, is proposed, and algorithms that almost attain this complexity are proposed. Some of them are based on a static allocation strategy (i.e. that does not depend on the rewards collected), and one of them is adaptive. First, this paper is not exactly the first on best arm identification in linear bandit problem. Indeed,* Hoffmann, Shahriari, De Freitas, On correlation and budget constraints in model-based bandit optimization with application to automatic machine learning (AISTATS 2014)consider a linear bandit model with gaussian noise. However, they study the fixed-budget setting, and the complexity term featured in their work involves a sum over all arms of a squared gap (that is, it does not really take into account the correlations induced by the linear structured).The theoretical results of the paper rely on Propositions 1 and 2 given in Section 2. The statement of these results are a bit imprecise, and it should be mentionned whether n is fixed or not. Indeed, the result extracted from [1] given as Proposition 2 should be P(\forall n\in\N, (2) holds) \geq 1-\delta, whereas Proposition 1 should be written as: for every n, for every fixed sequence x_n,P( (1) holds) \geq 1- \delta. This makes a difference, as an extra union bound over n might be required when using Proposition 1, as even when the sampling rule is static, the sampling rule is adaptive. Also a precise reference for Proposition 1 should be given. The terminology should be made more precise. In best arm identification, in the fixed-confidence setting, algorithms consists in two elements: a sampling rule and a stopping rule (and also a recommendation rule, but quite naturally, when stopping you always choose the best arm as if the current least-square estimate were the true parameter). The "static and adaptive strategies" mentioned in the paper relates only to the sampling strategy  as the stopping strategy of the G-allocation strategy is adaptive. In the proof of Theorem 1, a union bound over n seems to be missing (see above comment on Proposition 1): you need the display of line 674 to hold for all n, hence the union bound will bring you a log(CK^2/t^2\delta) for some constant C for example. Besides, in the statement of Theorem 1, 'beta-approximation' is not defined (neither in the text nor in Lemma 5 and 6).About the oracle: Section 3 is a bit confusing as when you have access to the parameter \theta^*, there is nothing left to do. I understand that you assume that the algorithms are based on confidence intervals and stop when one of the confidence intervals is contained on the C(x). You then study which shape these confidence intervals should have in order to be as quickly as possible included in C(x^*). Still the definition of the oracle appears quite imprecise (again if \theta^* was known the problem would be solved). Usually the definition of a complexity in best arm identification comes from a lower bound on the number of sample used, for any $\delta$-PAC algorithm: you don't really have such a result here and I don't see what justifies the claim "We characterize the complexity of the problem" in the introduction.  The algorithms are built on allocation strategies coming from experimental design theory (many of the results of Appendix B are adapted from [16]) which are hard to implement and must be approximated. It is not clear what is the overall numerical complexity of the proposed algorithm and how it scales with the different parameters. In the numerical experiments section, it is always frustrating to see comparison only between the methods introduced in the paper. As such, this section is not very informative. In particular, an obvious benchmark would have been to compare to an algorithm designed for best arm identification in classic (unstructured) bandit problems (LUCB, UGapE, etc.) This is particularly true as the problem chosen in the experiments is such that the arms are orthogonal, except for two of them; a situation which is very close to the classic d-armed bandit model. Comments about the choice of such a particular setting for the simulations would also be appreciable.Minor comments* There is an overlap in the notation: in the paragraph "the setting", \Delta is defined in two ways, \Delta(x) and \Delta(y) (one beeing functions of vectors, the others of directions): it might be more convenient for the reader to distinguish explicitly these two notions.* In Footnote 1 "used of all direction y": should it be "for all vector x"?* Line 232, "inthe" (should be separated).* In the boxed environment of page 4 (Figure 2), it should be Eq. 24 in place of Eq. 22 on the second line.* Figures 2 and 3 (which actually define the algorithms) refer to Equations that only appear in the supplementary material; please make the main paper self-contained.


Multi-Step Stochastic ADMM in High Dimensions: Applications to Sparse Optimization and Matrix Decomposition
In this paper, we consider a multi-step version of the stochastic ADMM method with efficient guarantees for high-dimensional problems. We first analyze the simple setting, where the optimization problem consists of a loss function and a single regularizer (e.g. sparse optimization), and then extend to the multi-block setting with multiple regularizers and multiple variables (e.g. matrix decomposition into sparse and low rank components). For the sparse optimization problem, our method achieves the minimax rate of O(s log d/T ) for s-sparse problems in d dimensions in T steps, and is thus, unimprovable by any method up to constant factors. For the matrix decomposition problem with a general loss function, we analyze the multi-step ADMM with multiple blocks. We establish O(1/T ) rate and efficient scaling as the size of matrix grows. For natural noise models (e.g. independent noise), our convergence rate is minimax-optimal. Thus, we establish tight convergence guarantees for multi-block ADMM in high dimensions. Experiments show that for both sparse optimization and matrix decomposition problems, our algorithm outperforms the state-of-the-art methods.

This paper proposed a multistep version of Stochastic ADMM method for high-dimension problems, in which there are multiple epochs defined. In each epoch, projections onto certain norm balls with decreasing radius are employed.As to the theoretical results, the authors showed that this simple variant achieves the minimax bound for sparse regression problem; while considering matrix decomposition problem, an excess risk bound, which is worse by a factor of d than the batch version, is obtained. Numerical simulation confirmed that the proposed algorithm performs better under both cases.The work is a little incremental considering that the proposed method is the combination of ST-ADMM with the method proposed in [5].  Both algorithms have the same algorithmic framework. In fact, the O(s \log d /T) excess risk bound is not very surprising, because O(1/T) comes from the doubling trick used in [5] and an earlier work [**][**] Elad Hazan, Satyen Kale, Beyond the regret minimization barrier: an optimal algorithm for stochastic strongly-convex optimization, COLT’11The factor s\logd is standard for high-dimensional problem with sparse structure. Furthermore, the authors mistakenly use regret bound to refer to excess risk bound through the paper, which is misleading.More surprisingly, the proposed method using stochastic ADMM outperforms [5] on sparse regression significantly in the empirical experiments, while they have the same order of excess rick bound. Note that sparse regression is a non-constrained problem in nature. Solving it using ADMM by introducing some auxiliary variable works better than directly solving the unconstrained problem using stochastic RDA is very counter-intuitive.Also, the organization of this paper should also be improved for better understanding. It is better to present a unified algorithm, and then give two examples, rather than presenting two algorithms separately for sparse regression and matrix decomposition.====The authors explained why their algorithm outperforms multi-step RDA in the rebuttal. I have no concern about the empirical experiments any more.


Stochastic Network Design in Bidirected Trees
We investigate the problem of stochastic network design in bidirected trees. In this problem, an underlying phenomenon (e.g., a behavior, rumor, or disease) starts at multiple sources in a tree and spreads in both directions along its edges. Actions can be taken to increase the probability of propagation on edges, and the goal is to maximize the total amount of spread away from all sources. Our main result is a rounded dynamic programming approach that leads to a fully polynomial-time approximation scheme (FPTAS), that is, an algorithm that can find (1− )-optimal solutions for any problem instance in time polynomial in the input size and 1/ . Our algorithm outperforms competing approaches on a motivating problem from computational sustainability to remove barriers in river networks to restore the health of aquatic ecosystems.

The paper generalize a previous work on stochastic network design on a directed rooted tree to model influence propagating from a single source to the whole network. In this paper, the directions on the edges are in both way and not directed away from the source as in the previous work. The paper generalizes the rounded dynamic programming approach from the previous work to get an FPTAS for the more general setting. The idea is to discretize the space of possible outcomes at any subtree. For each subtree rooted at a vertex v, there are 3 parameters defining the outcome: the influence within the subtree, the influence going through v to the subtree, and the influence going from the subtree through v to the rest of the tree. This crucially depends on the assumption that the reward factor/influence for 2 nodes s and t must be of the form h(s)*h(t) for a fixed function h. This assumption allows for aggregating influence going through v into 2 parameters as described above. The writing is clear and easy to follow. The dynamic programming approach is standard in the literature for designing FPTAS and in this particular case, quite similar to previous work. The setup is more general than in previous work but there is still the restriction that the reward factor for 2 nodes s and t must be of the form h(s)*h(t) for a fixed function h. It is only mentioned that this restriction is a good match for the application without any further explanation.== added after rebuttal ==Thanks for adding applications where the reward factors of the form h(s)*h(t) make sense.


Deep Recursive Neural Networks for Compositionality in Language
Recursive neural networks comprise a class of architecture that can operate on structured input. They have been previously successfully applied to model compositionality in natural language using parse-tree-based structural representations. Even though these architectures are deep in structure, they lack the capacity for hierarchical representation that exists in conventional deep feed-forward networks as well as in recently investigated deep recurrent neural networks. In this work we introduce a new architecture -a deep recursive neural network (deep RNN) -constructed by stacking multiple recursive layers. We evaluate the proposed model on the task of fine-grained sentiment classification. Our results show that deep RNNs outperform associated shallow counterparts that employ the same number of parameters. Furthermore, our approach outperforms previous baselines on the sentiment analysis task, including a multiplicative RNN variant as well as the recently introduced paragraph vectors, achieving new state-of-the-art results. We provide exploratory analyses of the effect of multiple layers and show that they capture different aspects of compositionality in language.

This paper proposes the use of 'deep' recursive networks by adding depth across space, which is motivated by the success of doing the same thing with recurrent networks. Each layer has its own parameters that are shared across different heights within that layer. The authors also propose the use of dropout and ReLUs, while fixing the word representations to pre-trained embeddings. Experimentation is performed on the sentiment treebank of Socher et al.The proposed method is novel although essentially a generalization of the same approach used with recurrent nets. The authors achieve very strong performance on the sentiment treebank dataset. The performance gains obtained with dropout and ReLUs are interesting in itself. That being said, some of the model and experimental design choices are questionable (or not well justified) and I believe some additional experimentation is necessary before I can wholeheartedly recommend acceptance.The paper itself is clearly written for the most part and sufficient detail is given for the reader to reproduce the results on their own.Detailed comments / questions:- line 138: What is W^eta and W^xh? These haven't been defined. Are these just to refer to either W^{eta}_L and W^{eta}_R? It would be clearer to have two equations: one for the inputs and another for the hidden layers. It would also help to summarize the parameter space i.e. {W_L, W_R, U, b, c}. Including the dimensionalities of each of your matricies when you define them would also be helpful.- lines 148-154: This seems very speculative. Is this based off of your own empirical results? If so, you should mention this. I'm not convinced that this is actually an issue. - baseline: The authors should include the results of "A convolutional neural network for modelling sentences" (Kalchbrenner et al) from ACL this year.- How come you didn't fine-tune the word vectors? Finetuning them on a sentiment task allows the words embeddings to become reflective of their sentiment. It seems strange to have phrases and sentence vectors that are discriminative of sentiment but where individual word vectors are not. (I realize this is OK with a deep RNN since you could just forward-pass the word vectors across layers). Is it possible that the improvements you're getting with deep RNNs happening because you are not fine-tuning the words? Conceivably, a single layer RNN with fine-tuned embeddings should do better. I think this needs to be controlled for.- line 251 (shared dropout units): What was the motivation for this choice?- lines 255-256 (stability): Alternatively, you could try constraining the norms of the weights or truncating the gradients, as is sometimes done with recurrent nets.- binary classification: This should be done separately. The issue is that at training time, you are also including the neutral class examples which deviates from the experiment protocol. Consequently, these results are not directly comparable to the existing approaches.- The analogy with deep recurrent nets is reasonable when you are making predictions at each node in the tree, as is the case on the treebank experiments. It would have been useful to see whether or not there is an advantage to using deep RNNs when only a single, root label exists. Would you still expect to see an improvement over a single layer RNN?


Sequence to Sequence Learning with Neural Networks
Deep Neural Networks (DNNs) are powerful models that have achieved excellent performance on difficult learning tasks. Although DNNs work well whenever large labeled training sets are available, they cannot be used to map sequences to sequences. In this paper, we present a general end-to-end approach to sequence learning that makes minimal assumptions on the sequence structure. Our method uses a multilayered Long Short-Term Memory (LSTM) to map the input sequence to a vector of a fixed dimensionality, and then another deep LSTM to decode the target sequence from the vector. Our main result is that on an English to French translation task from the WMT-14 dataset, the translations produced by the LSTM achieve a BLEU score of 34.8 on the entire test set, where the LSTM's BLEU score was penalized on out-of-vocabulary words. Additionally, the LSTM did not have difficulty on long sentences. For comparison, a phrase-based SMT system achieves a BLEU score of 33.3 on the same dataset. When we used the LSTM to rerank the 1000 hypotheses produced by the aforementioned SMT system, its BLEU score increases to 36.5, which is close to the previous state of the art. The LSTM also learned sensible phrase and sentence representations that are sensitive to word order and are relatively invariant to the active and the passive voice. Finally, we found that reversing the order of the words in all source sentences (but not target sentences) improved the LSTM's performance markedly, because doing so introduced many short term dependencies between the source and the target sentence which made the optimization problem easier.

SUMMARY:This paper introduces an LSTM-based approach to map arbitrary-length sequences to other arbitrary-length sequences, which is a required property in fully neural machine translation. The LSTM is trained to produce a fixed-size representation of a sentence, which then serves as the initial state of a second LSTM which generates a translated sentence. The authors evaluate their architecture on two machine translation datasets.GENERAL OPINION:The ideas represented in this work are extremely interesting, and I love the elegance and simplicity of the proposed RNN architecture. I am happy with the changes made in the second version in terms of performance.REMARKS:Figure 1: If I'm correct, the symbols W, X, Y, and Z are samples, and used as input for the next time step, right? If so, it might help to put arrows indicating this (output to input of next time step).line 88: What is BLUE point and BLUE score? A short explanation or reference would help.line 92: "a BLUE score of 28.w"lines 112 to 118: I'm somewhat sceptical that common RNNs would find it hard to model "long-term dependencies", as word sequences are not that long at all. Most sentences are not much longer than a dozen or so words, which falls well into the dependency length that can be modelled by RNNs. I do believe RNNs would be slower and harder to train though, so please rephrase that sentence.lines 138-139: Please refer to Alex Graves work on multi-layered LSTMs: A. Graves, A. Mohamed, G. Hinton. Speech Recognition with Deep Recurrent Neural Networks. ICASSP 2013, Vancouver, Canada.line 154: Please remain consistent with notations. "DE-> EN" is not much shorter than "German -> English". line 183: What is a beam search decoder? Reference? Explanation?lines 191-193: "any translation that has an UNK cannot possibly correct". Why not? Either I'm misunderstanding something here, or this sentence is wrong. If there is an UNK in the input sentence, it makes sense that there would be one in the output, no? Conversely, if there are no UNKs in the input dataset at all, there would be no reason to include UNKs at all. Please clarify.line 197: "Le Mans University"line 249: "more careful tuning"Figure 2: It would be more convincing to have some more examples. The ones presented now could be cherry-picked.


Universal Option Models
We consider the problem of learning models of options for real-time abstract planning, in the setting where reward functions can be specified at any time and their expected returns must be efficiently computed. We introduce a new model for an option that is independent of any reward function, called the universal option model (UOM). We prove that the UOM of an option can construct a traditional option model given a reward function, and also supports efficient computation of the option-conditional return. We extend the UOM to linear function approximation, and we show the UOM gives the TD solution of option returns and the value function of a policy over options. We provide a stochastic approximation algorithm for incrementally learning UOMs from data and prove its consistency. We demonstrate our method in two domains. The first domain is a real-time strategy game, where the controller must select the best game unit to accomplish a dynamically-specified task. The second domain is article recommendation, where each user query defines a new reward function and an article's relevance is the expected return from following a policy that follows the citations between articles. Our experiments show that UOMs are substantially more efficient than previously known methods for evaluating option returns and policies over options.

This paper proposes and analyses "Universal Option Models", an extension of discounted occupancy functions (Ng & Russell, 2000) to "options" (Sutton et al., 1999) – high-level actions that consist of a policy and a state-dependent termination probability function. After establishing a connection between an options discounted state occupancy function and an option rewards (Theorem 1) the application of options to learning and planning in large state spaces via linear function approximation is considered. Consistency and convergence results are given (Theorems 2, 3, and 4) before the efficacy of the technique relative to an earlier option-based technique is demonstrated on two domains.In general, the paper is clearly motivated and the main ideas are presented in a logical fashion. The experimental methodology is clearly explained and the results show a clear and significant improvement over the existing method by Sorg & Singh (2010). I am admittedly not well-versed in the reinforcement learning literature but this seems to be a strong point of the paper. However, I am surprised at the lack of related reinforcement learning work between 2000 and 2010. The references only cite work on PageRank (2002)  and citation networks (2008) in the intervening years.My main concerns surround the theoretical results in the paper. The notion of a "reward-less MDP" appears in Theorems 1 & 2 but is not formally defined anywhere. A theorem that rests on a definition in "scare quotes" makes me nervous as it is not entirely clear what the theorem applies to. It would be good to do so and provide a brief discussion of what is being captured by this definition.The use of a inner product between the vectors $u^o(s)$ and $r^\pi$ at line 151 is very confusing. Just before Definition 1, $u^o(s)$ is defined to be a vector indexed by states over the second argument of $u^o$ and $r^\pi$ is also a vector indexed by states. Accordingly, my interpretation of the inner product $(u^o(s))^\top r^\pi$ would be $\sum_{s'} u^o(s, s') r^\pi(s')$ which is *not* the same as the sum in (4) for $R^o(s)$. No part of the proof of Theorem 1 in the appendix does not use the inner product notation but instead shows the return of option $o$ satisfies (4) directly. As far as I can tell, the vector form of $R^o$ is not relied upon later in the paper so I do not believe this has serious implications for the rest of the paper.Theorem 4 refers to the "Robbins-Monro conditions" but, as far as I can tell these are not given or referenced anywhere in the paper. Even if these are common knowledge in RL it is poor practice to be this imprecise when stating theorems. Furthermore, the "proof" of this result is really only a sketch. Given that there is no page limit for appendices a full proof should be provided.


Using Convolutional Neural Networks to Recognize Rhythm Stimuli from Electroencephalography Recordings
Electroencephalography (EEG) recordings of rhythm perception might contain enough information to distinguish different rhythm types/genres or even identify the rhythms themselves. We apply convolutional neural networks (CNNs) to analyze and classify EEG data recorded within a rhythm perception study in Kigali, Rwanda which comprises 12 East African and 12 Western rhythmic stimuli -each presented in a loop for 32 seconds to 13 participants. We investigate the impact of the data representation and the pre-processing steps for this classification tasks and compare different network structures. Using CNNs, we are able to recognize individual rhythms from the EEG with a mean classification accuracy of 24.4% (chance level 4.17%) over all subjects by looking at less than three seconds from a single channel. Aggregating predictions for multiple channels, a mean accuracy of up to 50% can be achieved for individual subjects.

This paper uses Electroencephalography (EEG) recordings to study the possibility of determining the rhythm that a subject is listening to. The authors employ two different techniques: Stacked denoising autoencoders (SDA), and Convolutional Neural Network (CNN).The authors have generated EEG data through experimentation. 13 subjects have listened to 24 different rhythms. Each EEG recording consists of about 13 channels. Each rhythm was 32 seconds long and EEG sampling frequency was 400Hz. In one experiment, the authors train on the data from EEG signals of one individual and test on EEG data from the same individual. They use about 27 seconds of a 32 second rhythm for training and about 2.5 seconds for testing (exact time depends on rhythm tempo). They show that when trained and tested on a single individual, the accuracy of correctly classifying a rhythm into 24 classes is higher than chance (24.4% vs 4.17%). For one specific individual they can classify the EEG signal with 50.0% accuracy.In another experiment they train using the data from all subjects. In this example the accuracy drops to about 10% (still 4.17% chance). In the case of binary classification (whether East African or Western rhythm) the accuracy of joint classification is 57.1% (where chance is 50%). This shows that it is difficult to have a classification model that is appropriate for all subjects. This phenomenon is understandable and is shown in voice recognition literature as well.Although the dataset and the work are novel. There is little technical contribution in this paper. The paper primarily reports the results of two major algorithms on a new dataset. Although this is valuable on itself, I am not sure if it fits into the criteria of NIPS papers.


Blossom Tree Graphical Models
We combine the ideas behind trees and Gaussian graphical models to form a new nonparametric family of graphical models. Our approach is to attach nonparanormal "blossoms", with arbitrary graphs, to a collection of nonparametric trees. The tree edges are chosen to connect variables that most violate joint Gaussianity. The non-tree edges are partitioned into disjoint groups, and assigned to tree nodes using a nonparametric partial correlation statistic. A nonparanormal blossom is then "grown" for each group using established methods based on the graphical lasso. The result is a factorization with respect to the union of the tree branches and blossoms, defining a high-dimensional joint density that can be efficiently estimated and evaluated on test points. Theoretical properties and experiments with simulated and real data demonstrate the effectiveness of blossom trees.

The paper continues a line of work in graphical models to go beyondGaussian modeling of real-valued attributes. The apparent novelty isthe combination of a previous algorithm for non-parametric forestdensity estimation, which rests on a non-parametric generalization ofthe classical Chow-Liu algorithm for learning tree graphical models, with the standardgraphical LASSO (glasso), resulting on the authors call a "blossom treegraphical model." The paper states a type of statistical-consistency(theoretical) result, and a few experiments.* On Section 3: Can the authors' comment on how much the stated resultsays about *machine learning* properties (i.e., PAC orlarge-deviation, as opposed to data-size-asymptotic bounds)?  Can theauthors say anything at all about the properties of the constant N?What would be an expression of a lower or upper bound, even if exact?Just like the typical asymptotic convergence as the number of samplesgoes to infinity (e.g., CLT), it is hard to know when thelarge-deviation bounds of the kind provided in the paper start beingvalid. Without such information regarding N and the exact numberof samples n, while it may be an interesting and useful statisticalresult, it does not appear very useful to machine learning, IMHO.Hence, I would have Section 3 moved fully to the supplementarymaterial and replace it with more experimental evaluations. For example,I would have moved the sections Supplementary Simulations 1 and 2 intothe main body of the paper. But even then, a slightly more thoroughevaluation would be useful such as experimenting by changing the classor form of the underlying ground-truth density, and the class ofunderlying graphs, whenever applicable, the number of samples, and thenumber of attributes/variables. *  Given the nature of the paper, a reference to the Chow-Liu algorithm, from 1968, seems warranted, even if only in passing. C. K. Chow, and C. N. Liu, "Approximating Discrete ProbabilityDistributions with Dependence Trees," IEEE Transactions of InformationTheory, vol. IT-14, no. 3, May 1968Despite the different statistical setups/models (i.e., discretevs. continuous), and the fact that the work in the submitted papergoes beyond simple trees/forests/blossom representations, it isundeniable that the core of the underlying approach has its roots onChow-Liu's work.* It'd have been nice to see experiments on real-world data such asthose on microarray data that Liu et al (JMLR, 2011) performed. Isthe problem public access to the data? What about other datasets(e.g., fMRI)?* I am curious as the core distinctions among the following work andforest density estimation, which serves as the foundation for the submission?- Marwan Mattar, and Erik Learned-Miller.  Improved generative models forcontinuous image features through tree-structured non-parametricdistributions. UMass Amherst Technical Report 06-57, 10 pages, 2006.- Mihai Datcu, Farid Melgani, Andrea Piardi, and SebastianoB. Serpico, "Multisource Data Classification With Dependence Trees,"IEEE Transactions on Geoscience and Remote Sensing, vol. 40, no. 3,pp. 609-617, March 2002. I am interested in directly ML-relevant, as opposed to statistical,distinctions.* It has been quite a long time since Chow and Liu's work in 1968. Hence,I was originally skeptical that no one had actually tried anything like what theauthors' propose. If it exists, I have not been able to find it. So,while somewhat surprising, I must give the authors their due for thenovelty and originality of their proposed approach... Now, I maychange my mind later, should I find a reference :) ... All kidding aside,even if I find an old reference, it does not take much away from theauthors' originality/novelty, because no one in recent years haveproposed the technique, despite the intense attention to the problemof modeling continuous/real-valued attributes going back a decade, at least!


Restricted Boltzmann machines modeling human choice
We extend the multinomial logit model to represent some of the empirical phenomena that are frequently observed in the choices made by humans. These phenomena include the similarity effect, the attraction effect, and the compromise effect. We formally quantify the strength of these phenomena that can be represented by our choice model, which illuminates the flexibility of our choice model. We then show that our choice model can be represented as a restricted Boltzmann machine and that its parameters can be learned effectively from data. Our numerical experiments with real data of human choices suggest that we can train our choice model in such a way that it represents the typical phenomena of choice.

This paper presents how restricted Boltzmann machines can be used to learn human choice preferences. Because restricted Boltzmann machines do not force choices to be independent of the set of alternatives, the RBM model can represent typical human choice phenomena that cannot be modeled by the multinomial logistic models that are used as a standard model of human choice. The work proves that the RBM choice model can model several standard features of human choices, and that RBMs can flexibly model a range of strengths of these features. The evaluation includes synthetic data (in the supplementary material) as well as a comparison of RBM versus MLM models for a dataset on real human choices. Overall, the model seems technically sound and is indeed able to model typical human choice phenomena. One concern I had was whether the model is too flexible: are there features of human choices that this model would not be able to model, or parameter values that we might expect to share across datasets to constrain the learned parameters? The presentation does not seem to suggest that RBMs can be used as a cognitive model, but simply that they can predict human choices; this may make the concern about falsifiability less pressing, but it seems like some discussion of whether flexibility could be a negative feature of the model would be helpful.The theorems that are proven do demonstrate that given only a single hidden unit, the choice phenomena can be represented up to a particular range of strengths, but the evaluation is not restricted to using a single hidden unit and performance on test with a single hidden unit does not appear to be much better than not using a hidden unit. Since the theorems only show that a single typical choice phenomenon can be represented using one hidden unit, it makes sense that more hidden units may be required in the simulations (thank you for this clarification in the rebuttal). However, it seems that the discussion should then connect the theoretical and empirical work to make sense of what we can learn about human choice and how much we can generalize to new choices given the extreme flexibility of the actual models used in the simulations. One of my concerns related to the flexibility of the model was whether the RBM choice model could be used to predict choices about new items based on their features. This is addressed in the discussion with the sentence "The focus of this paper is in designing the fundamental structure of the RBM choice model and analyzing its fundamental properties, and the study about the RBM choice model with attributes is reported elsewhere." Thank you to the authors for clarifying that this is work in progress/not yet published - I think this sentence could be re-worded to make it more clear, but my concern about this paper standing on its own given the existence of the other work has been addressed. I do wonder, though, if it really makes sense to separate the two pieces of research. In the evaluation, I would have appreciated more qualitative explanation of the meaning of the hidden units in the RBM (i.e., why here 4 hidden units is approximately equivalent in performance to 16); such explanation might be helpful in understanding whether we can draw any conclusions about human choices based on the learned RBM model or if the model is only descriptive. To me, it seems like learning something about human choices (rather than just describing them computationally) needs to be a key outcome of this paper, especially given the flexibility of RBMs that the authors note.The qualitative clarification about generalizability in the rebuttal is helpful. I still believe it would strengthen the paper substantially to do more detailed testing than leave one out. In particular, it would strengthen the submission to more closely examine the effects of training set size and characteristics on test performance. Overall, my main concern about this paper is on its focus: computationally describing human choice phenomena and predicting a new choice from existing choices without looking at what we might learn about choices from this model or how much overfitting is a problem. Learning about choices from the model seems likely to be most interesting to computational cognitive scientists, but the current paper doesn't address this issue; I think it could be addressed using the current work, and I would appreciate seeing discussion of it. The overfitting concern seems to be most relevant for the broader machine learning community, which might be interested mainly in the applications. I think the paper could be substantially strengthened by addressing these issues or by explaining why they are not in scope; this concern is why I think presenting this work with the attribute work might result in more impact.Minor notes:line 18 (and later): "the choice made by humans" is used several times in the paper - I think this should read "the choices made by humans", but maybe this is a specialized use?line 375: Maglve20 -> Maglev20line 377: such attraction effect ->  such attraction effectsline 431: from given choice set -> from a given choice set


Stochastic Proximal Gradient Descent with Acceleration Techniques
Proximal gradient descent (PGD) and stochastic proximal gradient descent (SPGD) are popular methods for solving regularized risk minimization problems in machine learning and statistics. In this paper, we propose and analyze an accelerated variant of these methods in the mini-batch setting. This method incorporates two acceleration techniques: one is Nesterov's acceleration method, and the other is a variance reduction for the stochastic gradient. Accelerated proximal gradient descent (APG) and proximal stochastic variance reduction gradient (Prox-SVRG) are in a trade-off relationship. We show that our method, with the appropriate mini-batch size, achieves lower overall complexity than both APG and Prox-SVRG.

The paper presents an accelerated mini-batch randomized incremental gradient descent algorithm for composite minimization. The composite objective function is supposed to decompose into the sum of differentiable, Lipschitz-continuous terms and a proximal term. The contributions can be summarized as a two-fold extension of stochastic proximal variance reduction gradient descent (Prox-SVRG; Johnson&Zhang, Xiao and Zhang), using both an accelerated-gradient-type acceleration (Nesterov's) and a mini-batch setting. The obtained complexity estimate is in between  O((n+ alpha*kappa)log(1/epsilon))  and O((n+gamma*sqrt(kappa))log(1/epsilon)) depending on the appropriate choice of the mini-batch size wrt to the "optimal" (knowing the true condition number) mini-batch size (n is the number of terms in the diff.-part of the objective, kappa the condition number, and epsilon the accuracy). At worst, the proposed method performs as well as the accelerated proximal gradient decent and the Prox-SVRG. With appropriate mini-batch size, it potentially outperform both. Illustrative preliminary experiments are presented for learning with a multinomial logistic loss and an L1 regularization penalty. These experiments suggest that the proposed algorithm can outperform the competing methods with appropriate choice of the mini-batch size. Detailed comments:- the term "stochastic" is misleading here, since the composite optimization problem in Eq. 1 is completely deterministic and does not involve any expectation. Algorithms proceeding by randomly picking subset of terms in a large sum appearing the optimization objective are usually referred to "randomized gradient descent" algorithms. See [Bertsekas, "Nonlinear Programming", 2003] for reference in terminology. This stands in contrast to "stochastic gradient descent" algorithms whose purpose it to minimize an objective involving an expectation. - last of paragraph of page 2 is unclear, probably because of unfortunate copy-paste or edits; please correct. - throughout the paper, expectations are used, sometimes without conditioning to any filtration, sometimes with conditioning to a particular filtration, but always with the same notation. While the derivations seem to be correct, this is very confusing. Please use a less ambiguous notation, for instance E_t for E ( . | F_t). - Section 3 could start with a proof outline. Once the 4 lemmas are proved, the mechanics of the proof of Theorem 1 are quite clear and easy to follow. It would probably make sense to state the lemmas without proof (with proofs deferred to Appendix), and use the space to comment and illustrate what's going on in the proof. In particular, it would be interesting to include illustrations of the interpretation of the behaviour of the estimate sequences along the iterations for the convergence of the proposed algorithm. - Section 4 should be improved. A major issue of mini-batch approaches that limited their use in practice is the tuning of the mini-batch size. Several heuristics could be explored and compared here, since this parameter has clearly a major impact on relative performance wrt competing methods. It would also be interesting to compare the algorithms for several values of the regularization parameter. A default option could be the value that leads to the best validation error. It's quite shocking here (in a submission to a ML conference) that the value was set to a default value uniformly over all datasets, regardless of the generalization performance. Finally, it would be interesting to investigate the behaviour of the algorithms on a large dataset (n >> 1) with high-dimensional and dense features (p >>1, and dense vectors). This case is not covered by the datasets considered, but would clearly be worthwhile to consider in order to compare the algorithms.


Submodular Attribute Selection for Action Recognition in Video
In real-world action recognition problems, low-level features cannot adequately characterize the rich spatial-temporal structures in action videos. In this work, we encode actions based on attributes that describes actions as high-level concepts e.g., jump forward or motion in the air. We base our analysis on two types of action attributes. One type of action attributes is generated by humans. The second type is data-driven attributes, which are learned from data using dictionary learning methods. Attribute-based representation may exhibit high variance due to noisy and redundant attributes. We propose a discriminative and compact attribute-based representation by selecting a subset of discriminative attributes from a large attribute set. Three attribute selection criteria are proposed and formulated as a submodular optimization problem. A greedy optimization algorithm is presented and guaranteed to be at least (1-1/e)-approximation to the optimum. Experimental results on the Olympic Sports and UCF101 datasets demonstrate that the proposed attribute-based representation can significantly boost the performance of action recognition algorithms and outperform most recently proposed recognition approaches.

This paper presents an algorithm for attribute      (feature) selection, applied to activity recognition.  The      approach defines 3 criteria for choosing attributes for      recognition: high discrimination between pairwise classes, similarity      in the amount of discrimination over all pairs of classes, and      coverage of all pairs of classes.  The first two of these      criteria are formulated using a random walk approach, the latter      using a set-cover approach.  Finally, a greedy optimization      strategy is used to choose an attribute subset, starting from      the empty set.      This paper presents interesting research.  The main contribution of      the paper is the algorithm for feature selection.  This      algorithm seems novel, and is an interesting combination of      random walk, set-cover, and greedy optimization.  The      experimental results are also good -- the method shows      improvements over existing methods and baselines including no      feature selection and alternative strategies.  There are some      missing details in the experiments but the results seem solid.      My main concern about the paper is the motivation/intuition for      the approach -- the pieces seem chosen to try to use submodular      optimization.  The discussion on lines 133-145 describes 3      criteria.  I wasn't clear on the motivation for criterion 2 --      why can't some attributes be better for certain classes than      others?  Can't a final classifier choose combinations?  It seems      that this second criterion complicates the optimization, and      necessitates the use of the proposed techniques.      The resulting submodular optimization amounts to a      greedy approach.  It seems one could do similar greedy      optimization for other fomulations, for example ones with      criterion 2.  It would be interesting to know whether this      criterion is important empirically.      Overall, this concern is not fundamental; I think the paper is      very good, and should be accepted.      Other comments:      - I haven't gone over the details in supplementary material --      the intuitive explanations in the paper for these seemed      reasonable though.      - Typo "Combing"      - I didn't understand where the sparse coding (Eq. 5) is used in      the paper.  The experiments refer to KSVD for the DDA set.      Sparse coding of attributes doesn't seem to be a key component      of the paper, but if this is used it should be compared to      k-means or alternatives.  (Though there isn't space for this in      a conference version.)      - How are hyperparamters set in the experiments?  Are they tuned      by looking at the test (i.e. cross-validation) accuracies?      - The experimental results seem better than previous work.      Where are the numbers in Table 2 from?  [20] is shown with a      per-activity AP list, and mAP of 71.6.  In [20], Fig. 10(b)      shows mAP of 74.38%.  This would seem to be better than what is      in this paper.      - The related work section is thorough, though the work of Fu et      al., who also do similar attribute learning could be added:Learning multi-modal latent attributesY. Fu, T. Hospedales, T. Xiang and S. GongIEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI 2013)


Large-scale L-BFGS using MapReduce
L-BFGS has been applied as an effective parameter estimation method for various machine learning algorithms since 1980s. With an increasing demand to deal with massive instances and variables, it is important to scale up and parallelize L-BFGS effectively in a distributed system. In this paper, we study the problem of parallelizing the L-BFGS algorithm in large clusters of tens of thousands of shared-nothing commodity machines. First, we show that a naive implementation of L-BFGS using Map-Reduce requires either a significant amount of memory or a large number of map-reduce steps with negative performance impact. Second, we propose a new L-BFGS algorithm, called Vector-free L-BFGS, which avoids the expensive dot product operations in the two loop recursion and greatly improves computation efficiency with a great degree of parallelism. The algorithm scales very well and enables a variety of machine learning algorithms to handle a massive number of variables over large datasets. We prove the mathematical equivalence of the new Vector-free L-BFGS and demonstrate its excellent performance and scalability using real-world machine learning problems with billions of variables in production clusters.

This paper proposes a new vector free updating formulation for the two loop recursion in L-BFGS. The main insight is that the gradient direction can be expressed as a linear combination of historical state vectors. As a result, the dot product between the state vector and the gradient can be computed as a linear combination of dot products between state vectors which can be precomputed and independent of the size of input vectors. This results in a scalable and efficient implementation of L-BFGS in the  MapReduce framework. The proposed method is evaluated on a real-world large scale problem with billions of variables thus demonstrating its efficiency. It's also compared to hashing scheme and shows better accuracy.Overall, the proposed method is efficient and practical in large-scale settings. The paper is also well written. There are some typos as follows.- Line 82: "in contract to" -> "in contrast to"- Line 322: "compare the it" -> "compare it"


Subspace Embeddings for the Polynomial Kernel
Sketching is a powerful dimensionality reduction tool for accelerating statistical learning algorithms. However, its applicability has been limited to a certain extent since the crucial ingredient, the so-called oblivious subspace embedding, can only be applied to data spaces with an explicit representation as the column span or row span of a matrix, while in many settings learning is done in a high-dimensional space implicitly defined by the data matrix via a kernel transformation. We propose the first fast oblivious subspace embeddings that are able to embed a space induced by a non-linear kernel without explicitly mapping the data to the highdimensional space. In particular, we propose an embedding for mappings induced by the polynomial kernel. Using the subspace embeddings, we obtain the fastest known algorithms for computing an implicit low rank approximation of the higher-dimension mapping of the data matrix, and for computing an approximate kernel PCA of the data, as well as doing approximate kernel principal component regression.

The paper provides an algorithm to find a subspace embedding for polynomial kernel.The authors claim to propose the first fast OSE for non-linear kernel. This statement appears confusing to me. First of all, there are works on approximating OSE for non-linear kernel, for example [1] below, which does not mention in the paper at all. And the algorithm in [1] could be used for Gaussian kernel. Thus I would suppose that this paper focuses on the efficiency of the algorithm. Regarding the main theorem, it does provide a clear explanation that tensor sketch, in terms of \phi(v) T could be understood as an OSE. I do not manage to read the whole proof, but the intuition seems to be right. One of the main issues I have in mind is about the 5. in the Algorithm 1. The authors mention that \phi(v) T could be computed efficiently by [3]. However, it is not so intuitive for me since the original paper is not about Tensor Sketch directly. I suggest the authors provides more details on this, at least in the supplementary materials, since this is an important part of the algorithms. Regarding the efficiency, it would be interesting it the authors provide some comparisons between other methods for OSE.In the experiment section, the authors make some contradicting statement with the claim about efficiency of the algorithm. For example, "K-space can be rather expensive both in time and in memory to compute". This really confuses me very much, since as I understand one important point about the algorithm is that it gives a subspace embedding in polynomial time. Moreover, the experiments shows that the features extracted by the algorithm improves the performance of supervised learning compared to using the original features. This result makes sense, but it is already well-known from previous works that OSE usually gives better performances than original features. One interesting experiment I want to see would be the comparison between the proposed algorithm and the existing OSE algorithms.Please also be more careful about the mathematical formula in the paper. For example, at line 302 I should be "V U \phi(R) - \phi(A)" if I understand correctly.[1]  Kernelized Locality-Sensitive Hashing


Modeling Deep Temporal Dependencies with Recurrent "Grammar Cells"
We propose modeling time series by representing the transformations that take a frame at time t to a frame at time t+1. To this end we show how a bi-linear model of transformations, such as a gated autoencoder, can be turned into a recurrent network, by training it to predict future frames from the current one and the inferred transformation using backprop-through-time. We also show how stacking multiple layers of gating units in a recurrent pyramid makes it possible to represent the "syntax" of complicated time series, and that it can outperform standard recurrent neural networks in terms of prediction accuracy on a variety of tasks.

Review of submission 1062:Modeling sequences with a predictive gating networkSummary: The authors use BPTT to predict future observations from previous observations, and build a stack of such predictors. Applications: sine waves, rotating 3D objects. The approach outperforms certain traditional RNNs. Comments:An interesting approach, but relations to similar previous work are missing:1. Submission: "We show that we can build a deep model by stacking multiple layers of the transformation model, so that higher layers capture higher-oder transformations, in other words, transformations between transformations."The first very similar hierarchy of RNN predictors was described in 1992 [17]. Each RNN predicts its next input. Each is trained by BPTT. Higher-level RNNs see only inputs that lower levels could not predict. The approach in the submission seems very similar, although it is not quite the same - what exactly are the differences?[17] J. Schmidhuber. Learning complex, extended sequences using the principle of history compression, Neural Computation, 4(2):234-242, 1992One difference seems to be: "after extracting the transformation from two frames in the time series, the model predicts the next frame(s) by applying the transformation passed down by a higher layer (in lower layers of the model), or by assuming constancy of the transformation through time (in the top-most layer)."That is, unlike the apparently more general 1992 model, only 2 frames are used to predict the third?  Perhaps that's why the authors are drawing this analogy: "One may think of the bilinear model as performing a first-order Taylor approximation of the input sequence, where the hidden representation models the partial first-order derivatives of the inputs with respect to time." Another difference seems to be that the 1992 hierarchy compresses the observation sequence in unsupervised fashion, while the system of the authors does not. "More importantly, we use predictive training using backprop through time for our model, and show that this is crucial for achieving good prediction performance." This, however, also applies to the deep RNN hierarchy [17].The relation to the previous work should be made very clear!2. Submission says: "Multiplicative interactions (bilinear models) were recently shown to be useful in recurrent neural networks by [4]."In fact, ref [8] of the authors used multiplicative interactions for RNNs much earlier, with great success. 3. Experiments:Chirp signal: "The performances of the DPGM were compared with that of a standard RNN (trained with BPTT) and a CRBM (trained with contrastive divergence) [12] on a dataset containing chirps (sinusoidal waves that increase or decrease in frequency over time)."It is well known that standard RNNs trained by BPTT don't do well on (superimposed) sine waves, e.g., [18]. But there are RNN training algorithms where the connections to output units are trained by the pseudoinverse, and connections to hidden units are evolved to minimize the residual error. This works very well on (even superimposed) sine waves [18]. To compare to the state of the art, one should test against such systems.[18] J. Schmidhuber, D. Wierstra, M. Gagliolo, F. Gomez. Training Recurrent Networks by Evolino. Neural Computation, 19(3): 757-779, 2007The other experiments are interesting.4. Discussion"A major long-standing problem in sequence modeling is how to deal with long range correlations." Here one should cite the work [19] that described and analyzed this problem first:[19] S. Hochreiter. Diploma thesis, TUM, 1991.5. General comment: Interesting and publishable, but only provided the missing relations to similar previous work are made very clear.


Sampling for Inference in Probabilistic Models with Fast Bayesian Quadrature
We propose a novel sampling framework for inference in probabilistic models: an active learning approach that converges more quickly (in wall-clock time) than Markov chain Monte Carlo (MCMC) benchmarks. The central challenge in probabilistic inference is numerical integration, to average over ensembles of models or unknown (hyper-)parameters (for example to compute the marginal likelihood or a partition function). MCMC has provided approaches to numerical integration that deliver state-of-the-art inference, but can suffer from sample inefficiency and poor convergence diagnostics. Bayesian quadrature techniques offer a model-based solution to such problems, but their uptake has been hindered by prohibitive computation costs. We introduce a warped model for probabilistic integrands (likelihoods) that are known to be non-negative, permitting a cheap active learning scheme to optimally select sample locations. Our algorithm is demonstrated to offer faster convergence (in seconds) relative to simple Monte Carlo and annealed importance sampling on both synthetic and real-world examples.

Overview: this paper presents a fast alternative to MC methods for approximating intractable integrals.  The main idea behind Bayesian quadrature is to exploit assumptions and regularities in the likelihood surface, something which pure Monte Carlo ignores.  The authors in this paper model the square root of the integrand (the likelihood/prior) as a Gaussian Process.  Samples are then drawn according to some criterion - in this case, samples are chosen to the location of the maximal expected posterior variance of the integrand.  Intuitively, this is a location where the model knows the least about the value of the integrand, and stands to gain a lot of information.  Importantly, they stress the computational benefits of their BQ active sampling method over standard Monte Carlo techniques. The authors then approximate integrals for a variety of problems, including marginal likelihood calculation for GP regression and GP classification marginal likelihoods. Quality - This paper is technically sound: the problem is well motivated, the method is well described and their approach does a good job when compared to other methods for numerical integration.  Clarity- The paper is very well written and organized.   The authors do a good job conveying all aspects of the analysis.  They describe Bayesian Quadrature (and numerical integration in the first place), as well as existing approaches similar to theirs in a clear way.  They differentiate their own method by very clearly laying out their contributions.  They do a great job explaining their approach, and the process of going from problem to solution.  Originality- They present their method as a way to improve (both speed and some accuracy) existing methods for Bayesian quadrature.  They stress two contributions: the square root GP and ‘fast active sampling’.  The square root GP seems to be another way to model a positive function (the likelihood), and one that is typically less explored.  The authors also do a great job describing two ways to cope with the intractability of inference given a non-linear transformation of a GP (linearization and moment matching).  Significance-The authors describe an alternative tool to compute a marginal likelihood - an extremely difficult and important task.   The utility of such a tool is based on its speed, accuracy, and simplicity of implementation.  This paper lays out an alternative solution to this common problem - one that is competitive in speed, accuracy, and simplicity.  However, it remains unclear how significant this particular paper will be (or how much followup research it will inspire).  What are some future directions of research made possible by this contribution? Questions and comments: - Figure 1: In this example, are the hyperparameters of the GP learned?  If the covariance is something like a squared exponential, How does the length-scale cope with the crazy section?- Line 208: how does the log transform compare to the square root transform.  It seems somewhat clear that an unconstrained GP will more poorly, how well does another type of constrained GP perform? - Line 264: Why is the variance of the log transform worse (worse enough to make the whole scheme worse)? - Line 303: Is the task here just to integrate over the mixture of gaussians? - Fig 8+9: Maybe put the converged values in a table?  It’s hard to compare L, M and AIS here.


Convex Optimization Procedure for Clustering: Theoretical Revisit
In this paper, we present theoretical analysis of SON -a convex optimization procedure for clustering using a sum-of-norms (SON) regularization recently proposed in [8,10,11,17]. In particular, we show if the samples are drawn from two cubes, each being one cluster, then SON can provably identify the cluster membership provided that the distance between the two cubes is larger than a threshold which (linearly) depends on the size of the cube and the ratio of numbers of samples in each cluster. To the best of our knowledge, this paper is the first to provide a rigorous analysis to understand why and when SON works. We believe this may provide important insights to develop novel convex optimization based algorithms for clustering.

The paper revisits a convex optimization procedure for clustering andshows that when samples are drawn from two cubes that are sufficientlyseparated the method correctly recovers the cluster memberships fromthe data.  The paper contains a very interesting lifing technique thatis used in the proof.  The paper can be improved quite a bit -- bothin terms of the quality/clarity of the writing and the presentation of theproofs.  The originality is quite high, especially in the proof of the Theorem.  The technical details of the proof is quite frankly a bit tootedious, and I think can be considerably improved.  One possible suchimprovement is to note that the penalty term can be written \sum_i<j ||x_i-x_j|| = 1/2 \sum_i,j ||x_i-x_j||.  This should eliminate abunch of complicated sum manipulations.  I expect that the differenceoperator can now be replaced with the further lifted vector where allx_i-x_j are considered, so that D(E) can simply be described asD(E)=(E kron 1 - 1 kron E) and so forth.Also, note that the problem for p=1 is tractable and can very easily solved analytically for the two cluster case (without any lifting). Here are some other specific comments on the paper:Authors don't cite earliest occurence of method.  There is at least this earlier paper describing COP.Pelckmans K., De Brabanter J. Suykens J.A.K., De Moor B., Convex Clustering Shrinkage, in Workshop on Statistics and Optimization of Clustering Workshop (PASCAL), London, U.K., Jul. 2005. ftp://ftp.esat.kuleuven.ac.be/pub/SISTA/ida/reports/kp05-111.pdfIt's strange to call the NP-hardness of K-means an instability. Usually an instability would refer to small perturbations leading to big changes in the result?SON is previously used as an acronym for the method.  Why invent a new acronym (COP) in this paper?  That hardly seems to be properly respectful of the earlier work.a provable guarantee when COP succeeds in identifying cluster membership.  -- a provable guarantee that COP succeeds in identifying cluster membership under certain conditions?This sentence makes it sound like you can do more than your paper does.Isn't the operator D the same as D_1 up to a permutation?  Do you reallyneed D?The phrase "each cluster being a cube" is used repeatedly and isstrange.  I assume you just mean that the cluster corresponds to thepoints that were drawn from a uniform distribution over the cube.Rearrange the rows of A ...Note that such a rearrangement is not unique since the ordering of A^1and A^2 is not prescribed"for correctly determining" -- "to correctly determine"  or "forcorrect determination of" I have to give you compliements for proving Hocking's rotation conjecture.(3) seems overkill to state explicitly.  The space could be used in abetter way.The matrices on p.5 is presented with no intuition or clue as to whatthey are to be used for.  This is a case of putting the technicalmessy details in the paper and the intuition in the supplementalportion.  The appendix gives the intuition for the matrix \Omega, whereas the paper describes \Omega in gory detail with no sense of whyit's useful or where it comes from.  Can you switch the appendix andthe paper with respect to the \Omega description?What is V in equation (6).  This set has not been defined.Why are the Lemmas numbered starting with 2?line 403: "predict how the the performance" -- "predict how the preformance"Some additional comments for the Appendix/Supplement:It's strange for Lemma 1 to appear in the appendix.  Shouldn't this belemma 4 (or 3 if the others are numbered starting at 1)page 2 appendix.The statement after the sentence"Then, because A is column centered, we get"does not need the column centered property.This is only needed on line 084-086.line 164: column centerd -- column centeredThe statement on line 188 should be shown explicitlyBased on author feedback responding to my comments I have changed my review from "marginally above the acceptance threshold" to "Good paper, accept".


Tight Bounds for Influence in Diffusion Networks and Application to Bond Percolation and Epidemiology
In this paper, we derive theoretical bounds for the long-term influence of a node in an Independent Cascade Model (ICM). We relate these bounds to the spectral radius of a particular matrix and show that the behavior is sub-critical when this spectral radius is lower than 1. More specifically, we point out that, in general networks, the sub-critical regime behaves in O( √ n) where n is the size of the network, and that this upper bound is met for star-shaped networks. We apply our results to epidemiology and percolation on arbitrary networks, and derive a bound for the critical value beyond which a giant connected component arises. Finally, we show empirically the tightness of our bounds for a large family of networks.

The authors present novel theoretical upper bounds on the total numberof nodes that are eventually activated/influenced in the IndepnedentCascade (IC) model in terms of spectral properties of the "hazardmatrix" (defined in terms of the edge probabilities). They firstpresent a worst case bound where the initial influencers are chosen tomaximize the influence, and then present an average case bound wherethe initially influencers are chosen at random. In each case,the bounds demonstrate sub-critical and super-critical behavior. Inparticular, the sub-critical regime occurs when the spectral radius ofthe appropriate matrix is less than one, and gives bounds ofO(sqrt(n)) infected nodes in the worst case and O(1) in the averagecase. The show how their general results can apply to improveresults on SIR epidemic models and characterizing giant components inrandom graphs. They also give results for particular families ofgraphs and demonstrate the accuracy of their bounds empirically.The analysis of the paper is sophisticated and rigorous and givesdetailed upper bounds that seem to match or beat existing bounds in anumber of areas. The paper is well-written and clear.While the flavor of the results is not incredibly surprising---thereare many similar results relating giant components, epidemic spread,etc. to the spectral radius of some matrix---these results aresignificant because of their tightness, broad applicability, andquality of the analysis. They could potentially have algorithmicsignificance by providing upper bounds for the influence maximizationproblem (provided that the upper bounds are tighter than those givenby the approximation guarantee of the algorithm).The paper is atypical for NIPS in that it is a theoretical analysis ofa mathematical model without a real algorithmic component. Since itsheds light on models that are actively being researched by the NIPScommunity and makes interesting connections between those and othermodels I believe it is within scope.Comments / questions:* I believe the statement of Lemma 1 can be strenghtened to say that  the distribution of the final set of infected nodes is identical  under each of these processes (not just that the marginal  probability of each node i being infected is the same).* I think there is a minor problem with the proof of Lemma 1 in the  supplementary material. Eq. (1) gives a system of equations that  must be satisfied by the random variables {X_i}, but it does not  uniquely define the random process because there are also other  solutions to the set of equations. In particular, one can set  X_i = 1 for all nodes in some cycle that is *not* reachable from  from A and still satisfy these equations. So, showing that all three  processes give random variables that satisfy these equations doesn't  prove their equivalence. It should be easy to fix the proof, e.g.,  by using Eq. (8) instead as the characterization of the process.* The difference between O(sqrt(n)) and O(1) in the sub-critical  regimes for worst and average cases is interesting. Can you provide  any more intuition for this? Are there any other phase transitions  in the average case behavior?* Can you say anything about lower bounds in the super-critical cases?* The applicability of DTIC to undirected graphs in Lemma 3 was a  minor confusion DTIC was defined for directed graphs. The  supplementary material makes it clear that this this can be  interpreted as DTIC in the bidirected graph where p_{ij} =  p_{ji}. Perhaps this can be clarified in the main paper. * In the experiments, how are the edge probabilities set? What  parameter(s) are tuned to vary the spectral radius? More details  would help understand the experiments better.* Is each point in your plots the result of a single simulation or the  average over multiple trials? An average over multiple trials would  be a cleaner match to your bounds on the expected value. Error bars  would also help illustrate the variability. On that note (assuming  data points are individual trials), in Figure 3(a) it looks like the  variance of the number of infected nodes is higher in the  sub-critical regime. Can you say anything about this theoretically?


On the Computational Efficiency of Training Neural Networks
It is well-known that neural networks are computationally hard to train. On the other hand, in practice, modern day neural networks are trained efficiently using SGD and a variety of tricks that include different activation functions (e.g. ReLU), over-specification (i.e., train networks which are larger than needed), and regularization. In this paper we revisit the computational complexity of training neural networks from a modern perspective. We provide both positive and negative results, some of them yield new provably efficient and practical algorithms for training certain types of neural networks.

The paper considers computational and theoretical questions around training multi-layer neural networks. In particular, the authors consider, in some restricted settings, when and how efficiently good solutions may be obtained. Both positive and negative results are presented. The identification of appropriate settings under which a useful analysis can be undertaken is itself a result.Overall, the paper is clear and well-structured. Of the theory literature around neural networks, this effort is among the more grounded in reality, and more accessible. The paper includes several results which “sort of” flow together, giving the slightest hint of a manuscript that collected partially related pieces together into a whole. But, on balance the work is coherent enough. This reviewer did not check proofs in the supplementary material, however the general reasoning and flow of ideas presented in the paper itself appear to be sound. The theoretical results do seem interesting on the whole, although Thm 2 and Cor. 1 may be subsumed in existing results, or may be incremental in nature. The authors spend a good deal of time studying networks induced by the quadratic activation function, which is non-standard, however they crucially try to understand how networks with this nonlinearity might be compared to networks defined from more widely adopted nonlinearities (such as the sigmoid). Experimental validation is lacking: the authors ran out of space, and experiments are more of an afterthought. In a sub-field that is primarily driven by practical applications and experimental successes, experiments should probably feature prominently. One of the main contributions and selling-points of the paper is the “efficient algorithms” for training certain depth-2 and -3 networks, but it is hard to evaluate these methods. Limited experimental detail is provided, and only depth-2 networks are explored in a narrow context. The evidence given, however, appears positive. Weaknesses of the algorithms are not discussed in sufficient detail. More information about the experimental comparisons should be given, even if in the supplementary material, so that an interested reader can understand whether the baseline networks are realistic.In the absence of more information and a sharpened impact message/result, the “practical” algorithms suggested by the theory may be of  limited use. Inertia in this field seems strong, and many researchers might very well happily continue to use networks defined by sigmoid and rectification nonlinearities trained with vanilla  SGD, at any depth they prefer. In this sense, the theory results may be viewed as the dominant contributions by which the paper may be evaluated.Specific Comments:- Section 2: What are your assumptions on the network, and the basic properties of the map f?   The general mathematical setting lacks definition.- The assumption that  f(V) is continuous is also somewhat strong. Why do you need this?- Theorem 2 says that the net has to have at least as many hidden units as there are training points, opening the door to obvious memorization of the data. What is the practical usefulness of this theorem? Combined with the (brief) experiments in Sec 5, is this saying that we’re making it easier to memorize the data?- pg. 5, line 220: The classes of nnets defined by sigmoid and rectification nonlinearities are (proper) supersets of the \sigma_{0,1} class, so concluding from Corollary 1 that the former aren’t efficiently learnable doesn’t sound entirely helpful. Isn’t it precisely the networks that *aren’t* in \sigma_{0,1} which make the other two classes interesting? What can we say about, for instance, N_{\sigma_{sig}} \setdiff N_{\sigma_{0,1}} ?- pg. 5, line 227: In what sense is N_{\sigma_{sig}, L} less “expressive” than N_{\sigma_{0,1}} ? In the sense of universal approximation networks, or the narrower sense where the former is supposed to include the latter as L\to\infty?- Theorem 6: Does the result hold for any, unbounded n ? Don’t you need an asymptotic constraint on n dependent on d ? - Sec 4.3 seems reminiscent of Oja’s algorithm (Oja flows). How can it be compared to that literature?- More discussion about the computational complexity of the “modified” GECO algorithm is needed. In particular, the trace maximization required at each iteration, seems to potentially itself require many iterations (especially when there are many data points, or the problem lacks conditioning).- Experiments Section: The second experiment (overspecification) is lacking. Does the plot show the training error (MSE)? If so, it doesn’t seem surprising that a network with more units is able to reduce the training error faster -- but perhaps this experiment is meant to confirm the expected? One could argue that it is easier for the network to memorize the data. More importantly, overspecification typically goes hand-in-hand with some form of regularization, as the goal is not to minimize train set error as quickly as possible, but to quickly reach a solution with good generalization performance. It would help if the experiment could address this key question as it’s central goal, with confirmation that the train error drops suitably fast as more of a corollary.


Tighten after Relax: Minimax-Optimal Sparse PCA in Polynomial Time
We provide statistical and computational analysis of sparse Principal Component Analysis (PCA) in high dimensions. The sparse PCA problem is highly nonconvex in nature. Consequently, though its global solution attains the optimal statistical rate of convergence, such solution is computationally intractable to obtain. Meanwhile, although its convex relaxations are tractable to compute, they yield estimators with suboptimal statistical rates of convergence. On the other hand, existing nonconvex optimization procedures, such as greedy methods, lack statistical guarantees. In this paper, we propose a two-stage sparse PCA procedure that attains the optimal principal subspace estimator in polynomial time. The main stage employs a novel algorithm named sparse orthogonal iteration pursuit, which iteratively solves the underlying nonconvex problem. However, our analysis shows that this algorithm only has desired computational and statistical guarantees within a restricted region, namely the basin of attraction. To obtain the desired initial estimator that falls into this region, we solve a convex formulation of sparse PCA with early stopping. Under an integrated analytic framework, we simultaneously characterize the computational and statistical performance of this two-stage procedure. Computationally, our procedure converges at the rate of 1/ √ t within the initialization stage, and at a geometric rate within the main stage. Statistically, the final principal subspace estimator achieves the minimax-optimal statistical rate of convergence with respect to the sparsity level s * , dimension d and sample size n. Our procedure motivates a general paradigm of tackling nonconvex statistical learning problems with provable statistical guarantees.

In the paper, a sparse PCA algorithm was studied. At the beginning, the authors introduced existing methods and theirdrawbacks in terms of both statistical and computational properties. Then, a two stage algorithm was proposed toovercome the drawbacks. In the first stage of the algorithm, a computationally efficient but statistically inefficientmethod based on the ADMM is used to obtain a preliminary solution. In the second step, the algorithm called sparseorthogonal iteration pursuit is used to improve the solution obtained in the first stage. Also, some theoreticalproperties of the estimator were investigated. Numerical experiments showed the usefulness of the proposed methodscompared to some existing methods. This paper is well written, though constants are very dense. Especially, theoretical studies include some remarkableresults such as the minimax-optimality of the algorithm, the blessing of large sample size, and analysis for non-Gaussor dependent data. Though I could not read the supplementary of the paper in such a short view period, I'll try to check it out in theremaining review period, and give some comments.


Extracting Latent Structure From Multiple Interacting Neural Populations
Developments in neural recording technology are rapidly enabling the recording of populations of neurons in multiple brain areas simultaneously, as well as the identification of the types of neurons being recorded (e.g., excitatory vs. inhibitory). There is a growing need for statistical methods to study the interaction among multiple, labeled populations of neurons. Rather than attempting to identify direct interactions between neurons (where the number of interactions grows with the number of neurons squared), we propose to extract a smaller number of latent variables from each population and study how these latent variables interact. Specifically, we propose extensions to probabilistic canonical correlation analysis (pCCA) to capture the temporal structure of the latent variables, as well as to distinguish within-population dynamics from across-population interactions (termed Group Latent Auto-Regressive Analysis, gLARA). We then applied these methods to populations of neurons recorded simultaneously in visual areas V1 and V2, and found that gLARA provides a better description of the recordings than pCCA. This work provides a foundation for studying how multiple populations of neurons interact and how this interaction supports brain function.

It is becoming feasible to obtain cellular recordings with large numbers of cells simultaneously. In particular, this submission focuses on analysis when multiple populations of cells are recorded simultaneously (population being perhaps multiple brain areas, or cell types). In particular, they build latent variable models to describe the interactions between two, simultaneously recorded, neuronal populations.The baseline method for modelling interactions between populations with latent variables is pCCA. However, CCA only captures instantaneous interactions, and does not capture any temporal structure.The submission introduces AR-CCA, which defines a autoregressive linear-Guassian model on the latent state. In addition, they introduce gLARA which constrains the covariance structure by removing shared latent variables. They describe an expectation-maximization approach to fitting the model.The authors then examine the efficacy of the models for describing a neuronal recording of V1 and V2 (actually the residuals after subtracting the PSTH). First, they demonstrated that pCCA, which distinguishes between within-population and between-population covariances is able to capture the data with many fewer latent variables than factor analysis (which does not distinguish).Using cross-validation, they demonstrate that (unsurprisingly) AR-pCCA and gLARA, which contain dynamics are better able to model the data. gLARA is slightly better, which would indicate that the additional constraints in the model are helping to prevent overfitting. Finally, they examine the temporal structure of interactions between V1 and V2.QualityThe claims made are well-supported by the evidence. The choice of pCCA is a reasonable baseline for comparison.ClarityThe submission is well-written. In particular, the motivations behinds models and relationships between them is explained well which is very helpful for comprehension.OriginalityThe manuscript introduces a new model AR-CCA which combines CCA with dynamics to allow capture relationships in sequential data. This is original, and the authors demonstrate its efficacy (along with the more constrained gLARA) in capturing neuronal interactions.SignificanceThe submission is of primary interest to a specialised audience. While it is true that high-dimensional neuronal recordings are becoming increasingly common, the models presented are only relevant when multiple populations (known to the experimenter) are recorded simultaneously. Additionally, while the authors showed their models capture neuronal statistics (i.e. did well in cross-validation) it wasn’t clear what biological insights such models conveyed (for example, what does figure 5 help us understand about V1/V2 function). It would be helpful to at least speculate on this topic.


Recursive Context Propagation Network for Semantic Scene Labeling
We propose a deep feed-forward neural network architecture for pixel-wise semantic scene labeling. It uses a novel recursive neural network architecture for context propagation, referred to as rCPN. It first maps the local visual features into a semantic space followed by a bottom-up aggregation of local information into a global representation of the entire image. Then a top-down propagation of the aggregated information takes place that enhances the contextual information of each local feature. Therefore, the information from every location in the image is propagated to every other location. Experimental results on Stanford background and SIFT Flow datasets show that the proposed method outperforms previous approaches. It is also orders of magnitude faster than previous methods and takes only 0.07 seconds on a GPU for pixel-wise labeling of a 256 × 256 image starting from raw RGB pixel values, given the super-pixel mask that takes an additional 0.3 seconds using an off-the-shelf implementation.

In this paper, the authors propose a model called 'rCPN' composed of a sequence of neural networks for semantic scene labeling (label every pixel in a scene by the class it belongs to). They use the network proposed by Farabet et al. (multiscale CNN, with some small differences such as using ReLU non-linearity and Dropout) to extract features that describe local characteristic of superpixels. These local featuresare then passed through a network to map them into a semantic space where local information can be propagated. This local information is then recursively aggregated using a parse tree hierarchy. Once the scene is completely aggregated, a third network is used to disseminate the holistic description back into the initial superpixel segmentation. Finally, a fourth network is used to classify the superpixels into different classes.The training is done into two separate stages: first the mutliscale CNN is trained from raw pixel. Then the image is segmented into a set of superpixels, and binary parse trees are made. Finally, rCPN (consisting of the four networks) are trained to predict the label of each superpixel.The paper is well written in general and the ideas are easy to get. The Related Works section could be inserted between Introduction and Learning sections instead. It would be interesting to make a better comparison with other models also using neural networks in the Related Works section (tradeoff between improved results/complexity of model). For the Experiments sections, the Stanford Background dataset results could be introduced before the SIFT Flow dataset ones, as it is a simpler dataset somehow. In line 259, when comparing the training time with Farabet et al you should mention you do GPU training while Farabet et al. explicitly mention no use of GPU. Also, in Table 1, you add the inference time for Pinheiro et al which I couldn't find in the cited paper, maybe you should just leave NA as in the others.The authors achieve good results with the expense of a complex, non-straightforward model (many different steps during the training).For the experiments, maybe it would be interesting to mention the other results from Farabet (pure multi-CNN and multi-CNN/superpixel) and Pinheiro (considering smaller output resolution) with faster models, which achieve relatively good results with faster inference time (and a more straightforward models trained from raw data in one step). Also, how many parameters does the model has, compared to other approaches?


New Rules for Domain Independent Lifted MAP Inference
Lifted inference algorithms for probabilistic first-order logic frameworks such as Markov logic networks (MLNs) have received significant attention in recent years. These algorithms use so called lifting rules to identify symmetries in the first-order representation and reduce the inference problem over a large probabilistic model to an inference problem over a much smaller model. In this paper, we present two new lifting rules, which enable fast MAP inference in a large class of MLNs. Our first rule uses the concept of single occurrence equivalence class of logical variables, which we define in the paper. The rule states that the MAP assignment over an MLN can be recovered from a much smaller MLN, in which each logical variable in each single occurrence equivalence class is replaced by a constant (i.e., an object in the domain of the variable). Our second rule states that we can safely remove a subset of formulas from the MLN if all equivalence classes of variables in the remaining MLN are single occurrence and all formulas in the subset are tautology (i.e., evaluate to true) at extremes (i.e., assignments with identical truth value for groundings of a predicate). We prove that our two new rules are sound and demonstrate via a detailed experimental evaluation that our approach is superior in terms of scalability and MAP solution quality to the state of the art approaches.

Summary:This paper proposes two rules for lifted inference in MLNs by exploiting the structure for MAP queries.The first rule is for applying MAP inference over a reduced theory such that single occurrences in equivalence classes are reduced to a unary sized domain. The second rule applies to the single occurrence theory, ignoring the tautology formulas for extreme assignments. Quality:This paper has a good quality. All theorems and lemmas have proofs in the paper or in the supported document. While the experimental results seem very promising, they are confusing with respect to the NSLGRB approach (Sarkhel et. al paper). In their paper, the results show constant growth w.r.t the domain size and time, however the pictures here show that the cost increases in time and domain size. Is there any justification of how the results presented in this paper are different from the claim of the original paper? Maybe the number of domain sizes (100 in their claims vs 500)? I am not convinced that the results are very different from Sarkhel's method. Clarity:The paper is well-organized and each section serves its purposes. There are nice definition such as the definition of single occurrence and extreme assignment.However in some places there is difficulty understanding all the theorems without explicit examples. For instance, section 4.1. requires examples mentioned in the text not in the additional document. Without this example it is a bit difficult to understand the rest of this section. Also in Algorithm2, the function getAllTautologyAtExtremes adds no new information and maybe isTautologyAtExtremes should have been explained in pseudo code instead. Originality:The originality of this paper is its major issue. First rule is essentially breaking the theory into sub-theories, assigning new weights according to their presence in an equivalence class. It seems this approach is not that different than previous network simplifications such as the decomposer. According to the proof of Theorem 4.2 a decomposer has the same definition of a single occurrence of an equivalence class. It seems that the decomposition idea in this work is very similar to the decomposer idea, only at a different level (on the MAP inference applied over a MLN). While the assignment of the new weights is different and novel but as claimed by corollary 4.1 this assignment is to some extend intuitive.Similarly the second rule is exactly derived from the paper of Sarkhel et. al. and thus is not considered a new innovative rule on its own. I would argue that the idea in this work is a modification of previous rules on MLN networks (specifically Sarkhel et. al) now applied to the MAP queries. There is not enough motivation to why this work and its ideas are novel.  Significance:According to the mentioned reasons this paper has minor significance. Due to the fact that its data are all from previous work or benchmarks, the results are not clear ( mentioned before) and the authors have failed to mention what happens to the cases where non of the formulas are single occurrences: there is a strong restriction on the theory. The major claim of independence is also mentioned in previous work and thus is not a new result for MLNs.


Near-Optimal Density Estimation in Near-Linear Time Using Variable-Width Histograms
Let p be an unknown and arbitrary probability distribution over [0, 1). We consider the problem of density estimation, in which a learning algorithm is given i.i.d. draws from p and must (with high probability) output a hypothesis distribution that is close to p. The main contribution of this paper is a highly efficient density estimation algorithm for learning using a variable-width histogram, i.e., a hypothesis distribution with a piecewise constant probability density function. In more detail, for any k and ", we give an algorithm that makesÕ(k/" 2 ) draws from p, runs inÕ(k/" 2 ) time, and outputs a hypothesis distribution h that is piecewise constant with O(k log 2
(1/")) pieces. With high probability the hypothesis
, where d TV denotes the total variation distance (statistical distance), C is a universal constant, and opt k (p) is the smallest total variation distance between p and any k-piecewise constant distribution. The sample size and running time of our algorithm are optimal up to logarithmic factors. The "approximation factor" C in our result is inherent in the problem, as we prove that no algorithm with sample size bounded in terms of k and " can achieve C < 2 regardless of what kind of hypothesis distribution it uses.

Near-optimal density estimation in near-linear time using variable-width histogramsGiven a univariate distribution with support on the unit interval, this paper develops a method for estimating the density of the distribution via a histogram. The histogram has the following features: the bins are data-dependent, the bins are determined efficiently (near linear time), the number of bins k is specified by the user, and the estimator satisfies a "probably approximately correct" type performance guarantee. The paper is entirely theoretical. Pros: The writing is clear. I believe the results to be novel and theoretically pretty interesting. Cons:  Some experiments would not have hurt the paper. A drawback of the theoretical framework is that model order k is not adaptively chosen.My main quibble is with the omission of some relevant references. In particular, the algorithm developed by this paper has a course-to-fine kind of strategy, and harkens to multiscale and wavelet-based methods for density estimation. Here are two relevant references:Rebecca Willett and Robert Nowak, "Multiscale Poisson Intensity and Density Estimation" IEEE Transactions on Information Theory, vol. 53, no. 9, pp. 3171-3187, 2007.J. Klemelä. (2009). Multivariate histograms with data-dependent partitions. Statistica Sinica. 19(1): 159-176. I would recommend the authors do both forward and reverse citation searches starting with these two papers. While this literature does not express its results in the PAC setting, and there is no doubt the above works pursue a different computational strategy, I think there are nonetheless some strong similarities. In fact, the above papers go beyond the submitted paper in looking at piecewise polynomial estimates (for smoothness classes beyond piecewise constant), adaptivity to model order, multiple dimensions, all while attaining optimal performance with efficient algorithms. I don't know if those results can be translated to give an apples-to-apples comparison with the submitted work, but if so, I would find that really interesting.Other comments:You say that you are restricting to the unit interval "without loss of generality." Yet, it's not clear whether your technique extends to densities with unbounded support.At line 163, what do you mean that "the L_1 distance is a stronger metric". By Holder's inequality, and since your domain is the unit interval, we know that the L1 norm is bounded by the L2 norm, but wouldn't you need to reverse direction for your statement to make sense? I'm possibly forgetting some basic fact here.The first paragraph of the "preliminaries" section confuses me on one point. You say that "for convenience in the rest of the paper we work with the L_1 distance." Yet you are also allowing distributions with point masses, in which case the distribution does not have a Lebesgue density. If you are allowing point masses, I think you need to stick to TV, and when you say density, be clear about the dominating measure. If you are allowing mixed continuous-discrete distributions, the term "density" seems inappropriate. You're estimating the probability measure.The term "semi-agnostic learning" in the title of subsection 3.2 is not defined.In the last line of the statement of theorem 6, I think you mean q' instead of q.