(More) Efficient Reinforcement Learning via Posterior Sampling
Most provably-efficient reinforcement learning algorithms introduce optimism about poorly-understood states and actions to encourage exploration. We study an alternative approach for efficient exploration: posterior sampling for reinforcement learning (PSRL). This algorithm proceeds in repeated episodes of known duration. At the start of each episode, PSRL updates a prior distribution over Markov decision processes and takes one sample from this posterior. PSRL then follows the policy that is optimal for this sample during the episode. The algorithm is conceptually simple, computationally efficient and allows an agent to encode prior knowledge in a natural way. We establish anÕ(τ S √ AT ) bound on expected regret, where T is time, τ is the episode length and S and A are the cardinalities of the state and action spaces. This bound is one of the first for an algorithm not based on optimism, and close to the state of the art for any reinforcement learning algorithm. We show through simulation that PSRL significantly outperforms existing algorithms with similar regret bounds.

Summary: this paper gives a regret analysis for an
      episodic reinforcement learning motivated by posterior sampling (as known
      as Thompson sampling). The idea is to sample, from the posterior, an MDP
      model at the beginning of each episode, follow this MDP's optimal policy
      in that episode, update the model posterior, and then repeat. In the past,
      despite good empirical performance, this algorithm (called PSRL) remains a
      heuristic approach without strong theoretical guarantees. This paper
      provides the first regret analysis, with a bound of O(\tau * S * \sqrt{A *
      T}), where H is the length of the episode and T is the total number of
      stpes. Numerical experiments on very simple MDPs show PSRL significantly
      outperform the other algorithm UCRL2 that enjoys a similar regret bound.
      * Quality: this paper makes an interesting theoretical
      contribution to RL in finite-state, finite-action, episodic MDPs. Building
      on recent advances in the multi-armed bandit literature, it is a
      successful step towards understanding finite-sample performance of
      Bayesian-style RL algorithms. * Clarity: the paper is written very
      clearly. Despite the heavy technical details, the basic ideas are
      explained well and intuitively. * Originality: The algorithm PSRL
      is not new; instead, the novelty in the paper is in its (first) regret
      analysis. The key in the analysis is Lemma 2, which equates the *expected*
      regret with the regret in the sampled MDP (assuming the MDP is sampled
      from the posterior). A similar observation has been used to proved
      expected regret bounds of posterior sampling in multi-armed bandits [15].
      While the rest of the analysis seems pretty standard in reinforcement
      learning, the application of Lemma 2 in this kind of analysis, and
      especially for expected regret, appears sufficiently novel. *
      Significance: Posterior sampling has been very successful in multi-armed
      bandits that raised quite a lot of interests in its theoretical study
      recently. This paper is the first that extends analytic ideas to MDPs and
      shows strong regret bounds of posterior sampling. Not only is the result
      the first of its kind, it may generate further interests in the RL
      community to investigate a new class of algorithms based on posterior
      sampling. Detailed comments: * While the analysis and
      bounds are very interesting, the paper should make it clear that the
      nature of the bounds here is fundamentally different from that in the
      literature. In particular, existing bounds are worst-case bounds, while
      the bounds here are average-case (averaged by the prior/posterior). It is
      the averaging that makes the key Lemma 2 possible. For the same reason, it
      seems the paper's *expected* bounds may be weaker than worst-case bounds.
      I hope the authors can comment on these points in the paper. * In
      the paper, UCRL2 is the only algorithm to compare against PSRL.
      Justifications are needed for this choice. PSRL, in the current
      description in the paper, applies to episodic tasks and aims to optimize
      undiscounted, finite-horizon total reward. In contrast, UCRL2 applies to
      continuing tasks and aims to optimize average reward. As a consequence,
      their regret bounds are not directly comparable: UCRL2 depends on a
      diameter parameter, while PSRL has the horizon length in the bound. There
      may be a way to relate them, but it is not obvious how. * Instead
      of UCRL2, another piece of work may be more relevant: Claude-Nicolas
      Fiechter: Efficient Reinforcement Learning. COLT 1994: 88-97 Although
      the Fiechter paper considers the PAC framework, it considers
      finite-horizon problems, so seems relevant to the present work. *
      The simple experiments show PSRL is empirically much better than UCRL2. It
      does not strike me as a surprise, given the very conservative nature of
      UCRL2. And, because of the gap the kinds of problems the two algorithms
      are designed for (see a related comment above), the empirical comparison
      carries even less information. * Line 255, 'depends only the'
      --> 'depends only on the' * Line 278, please be specific what
      the 'high probability' is. * Line 411, is it too restrictive to
      assume that the optimal average reward is uncorrelated with episodic
      length?


When Are Overcomplete Topic Models Identifiable? Uniqueness of Tensor Tucker Decompositions with Structured Sparsity
Overcomplete latent representations have been very popular for unsupervised feature learning in recent years. In this paper, we specify which overcomplete models can be identified given observable moments of a certain order. We consider probabilistic admixture or topic models in the overcomplete regime, where the number of latent topics can greatly exceed the size of the observed word vocabulary. While general overcomplete topic models are not identifiable, we establish generic identifiability under a constraint, referred to as topic persistence. Our sufficient conditions for identifiability involve a novel set of "higher order" expansion conditions on the topic-word matrix or the population structure of the model. This set of higher-order expansion conditions allow for overcomplete models, and require the existence of a perfect matching from latent topics to higher order observed words. We establish that random structured topic models are identifiable w.h.p. in the overcomplete regime. Our identifiability results allow for general (non-degenerate) distributions for modeling the topic proportions, and thus, we can handle arbitrarily correlated topics in our framework. Our identifiability results imply uniqueness of a class of tensor decompositions with structured sparsity which is contained in the class of Tucker decompositions, but is more general than the Candecomp/Parafac (CP) decomposition.

The paper is concerned with identifiability of latent
      variable topic models, characterizing when the latent structure is
      identifiable from observable moments in these models. The authors
      introduce a multi-view type of model called the n-persistent topic model
      where each set of n words (in sequence) share the same topic and study how
      n plays a role in identifiability. The first theorem is a
      deterministic result providing sufficient conditions for identifying the
      population structure (or the topic distributions) from the (2rn)-th
      observable moment. The three conditions involve non-degeneracy of the
      prior distribution over topic probability, an non-overlapping condition on
      the sparsity pattern of the columns of the population structure matrix,
      and a condition relating the kruskal rank of the population matrix to the
      sparsity of each column. The paper then analyzes population
      structures with random sparsity pattern and characterizes parameter
      settings (sparsity, degree of overcompleteness) under which these models
      are w.h.p. identifiable. Here, the authors show that if the number of
      topics is O(p^n) where p is the vocabulary size and n is the persistence
      parameter, and the support of each topic distribution is O(p^{1/n}), then
      these models are w.h.p. identifiable. While the paper deals with
      matrix representations throughout, they comment that working with the
      tensor representations would yield uniqueness conditions for a range of
      tensor decompositions (depending on the persistence parameter). This is an
      interesting result which is presented as an afterthought in the paper but
      should be emphasized. Working with the tensor representations may also
      lend to readability. The results of the paper are novel and
      interesting. The paper, however, suffers in terms of readability; some
      more intuition about the conditions and the main theorem would
      significantly improve the clarity of the paper. For example, condition 2
      and 3 together imply that ideal matrices have high degree of sparsity
      while still containing a n-perfect matching but it's not obvious why these
      matrices are the ideal ones. Moreover, the awkward matrix notation makes
      the paper significantly harder to read. Questions: 1. For
      condition 1 to be satisfied, it is essential that the topic proportions is
      a random quantity, but this is not always assumed in analysis for latent
      variable models (if h is fixed then M_{2r}(h) is always rank one). Why is
      this condition required? Previous analysis [9] seem to not require a prior
      distribution on the topic proportions while still guaranteeing
      identifiability.


One-shot learning by inverting a compositional causal process
People can learn a new visual class from just one example, yet machine learning algorithms typically require hundreds or thousands of examples to tackle the same problems. Here we present a Hierarchical Bayesian model based on compositionality and causality that can learn a wide range of natural (although simple) visual concepts, generalizing in human-like ways from just one image. We evaluated performance on a challenging one-shot classification task, where our model achieved a human-level error rate while substantially outperforming two deep learning models. We also tested the model on another conceptual task, generating new examples, by using a "visual Turing test" to show that our model produces human-like performance.

Reviewer response to rebuttal: I have read
      through the author's rebuttal and I am happy with the proposed changes. I
      have not changed my review as I already recommended this paper for
      acceptance. Previous Review: In this work, the authors
      develop a hierarchical generative model for producing and classifying
      written characters with the goal of achieving a high level of performance
      with just one training example. The model is rooted in learning the
      compositional structure of characters and the causal relationship that
      dictates how characters are produced. The model is compared to a simpler
      version of the model that does not represent character strokes, a deep
      boltzmann machine approach, and a hierarchical deep learning method. In
      addition to significantly outperforming these comparison models, the model
      achieves an impressive error rate that is very similar to human
      classification error rate. Furthermore, the authors demonstrate that the
      model can generate realistic character drawings after receiving only one
      training example as tested by a “Visual Turing Test” where participants
      were unable to reliably distinguish human drawn characters from machine
      drawn characters. The success of this approach appears to be attributed to
      its ability to learn from the richer compositional and causal properties
      of one image. COMMENTS TO AUTHORS: The near-human
      level of performance in a one-shot learning task in this work is very
      impressive and suggests that there is potentially great value in
      exploiting compositionality and causality in this domain. My only
      complaint is that the model is very complex which makes it a bit difficult
      to identify where the big win is. However, the comparison to the similar
      Affine model helps with this problem. It would be nice to see this
      model compared directly to results presented in previous work (maybe
      [25]). MINOR COMMENTS: I had trouble finding reference
      [25] (R. Salakhutdinov, J. B. Tenenbaum, and A. Torralba. Hierarchical
      deep models for one-shot learning. In Neural Information Processing
      Systems (NIPS), 2011.) Perhaps one of these was the intended
      reference? R. Salakhutdinov, J. Tenenbaum , A. Torralba. Learning
      to Learn with Compound Hierarchical-Deep Models. NIPS, 2011. OR
      R. Salakhutdinov, J. Tenenbaum, and A. Torralba . One-Shot
      Learning with a Hierarchical Nonparametric Bayesian Model. JMLR WC&P
      Unsupervised and Transfer Learning, 2012.


Efficient Supervised Sparse Analysis and Synthesis Operators
In this paper, we propose a new computationally efficient framework for learning sparse models. We formulate a unified approach that contains as particular cases models promoting sparse synthesis and analysis type of priors, and mixtures thereof. The supervised training of the proposed model is formulated as a bilevel optimization problem, in which the operators are optimized to achieve the best possible performance on a specific task, e.g., reconstruction or classification. By restricting the operators to be shift invariant, our approach can be thought as a way of learning sparsity-promoting convolutional operators. Leveraging recent ideas on fast trainable regressors designed to approximate exact sparse codes, we propose a way of constructing feed-forward networks capable of approximating the learned models at a fraction of the computational cost of exact solvers. In the shift-invariant case, this leads to a principled way of constructing a form of taskspecific convolutional networks. We illustrate the proposed models on several experiments in music analysis and image processing applications.

Summary of paper: This is a technical paper on
      supervised sparse coding. It defines a generalized form of Lasso problem
      that equates some input variables x (e.g. the input image) to some hidden
      variables y (e.g. the underlying, analyzed or denoised form of the image)
      in least squares via two arbitrary weight matrices -- ||M1*x-M2*y||^2 --
      adding an L1 penalty on a linear mapping Omega*y and an optional L2
      penalty on y itself. This formulation is general enough to cover both
      analysis and synthesis (generative) problems. The paper solves this system
      for y(x) via a proximal iteration on auxilliary variables z = Omega*y --
      an ADMM style method -- and it also offers fast approximation of y(x) via
      a network that contains an unwound, truncated ADMM with re-learned
      parameters along the lines of Gregor & LeCun [11]. Finally, it
      proposes supervised parameter (mapping matrix) learning using stochastic
      gradient descent over an arbitrary problem-specific loss function on y(x),
      and to this end it derives some explicit formulae for cost gradients in
      terms of sign(Omega*y) and the matrices and auxilliary vectors involved.
      The method is illustrated on an image super-resolution problem and a
      polyphonic music transcription one. It seems to give quite good results on
      both, although I found this aspect difficult to judge and would bow to
      more expert opinions on this. Review: This is a reasonable but
      somewhat pedestrian technical paper. AFAIK the precise model studied is
      new, and I agree that it is useful to have very general models of this
      kind as references and tutorial examples, especially as there are now so
      many variant models in L1 learning that it is difficult to keep track of
      all of them. However the techniques used -- proximal projection, ADMM,
      Gregor-LeCun approximation network, SGD, L2 smoothing or limiting
      arguments for the local gradient/subgradient estimates -- are all
      straightforward and fairly standard for recent methods, and the
      derivations themselves contain no surprises. Also, for many of the most
      interesting special cases, more specialized algorithms are likely to be
      preferred: the iterations for a model this general are necessarily rather
      expensive (equations of size = number of image pixels need to be resolved,
      at least in principle) and depending on the exact M1, M2, Omega,
      potentially also very ill-conditioned. Still, on balance it will be useful
      to have this study recorded, so I am favourable to acceptance if space
      permits. Further points: - In the first equation of
      proposition 1, "A" should presumably be "P_{Lambda^c}". The derivation of
      the gradient could perhaps be relegated to an appendix. Fig.1 defines
      different A,U,B matrices - consider changing notation there to avoid
      confusion. - More information on the effect of K in SI-NN, on the
      (especially ambiguous?) cases where SI-NN fails but full SI-ADNN doesn't,
      and on the degree to which the specific loss influences the learned
      encoders would be useful. - Just writing "matrix inverse" in Q, B,
      etc, is not very satisfying. These very large sets of equations are
      presumably solved by conjugate gradients or similar, and this will fail if
      the matrices become too ill-conditioned. Comment on the methods actually
      used.


Sign Cauchy Projections and Chi-Square Kernel
The method of stable random projections is useful for efficiently approximating the l α distance (0 < α ≤ 2) in high dimension and it is naturally suitable for data streams. In this paper, we propose to use only the signs of the projected data and we analyze the probability of collision (i.e., when the two signs differ). Interestingly, when α = 1 (i.e., Cauchy random projections), we show that the probability of collision can be accurately approximated as functions of the chi-square (χ 2 ) similarity. In text and vision applications, the χ 2 similarity is a popular measure when the features are generated from histograms (which are a typical example of data streams). Experiments confirm that the proposed method is promising for large-scale learning applications. The full paper is available at arXiv:1308.1009.
There are many future research problems. For example, when α → 0, the collision probability is a function of the resemblance (of the binary-quantized data). This provides an effective mechanism for resemblance estimation in data streams.

This paper proposes to use a signed Cauchy random
      projection to approximate the chi-square kernel with relatively low
      errors. The motivation comes from generalizing the Sim-hash algorithm and
      noting that chi-square is related to the collision probability.
      Overall, the reviewer finds the finding quite interesting and
      innovative, as no previous attempts seem to have linked chi-square with
      random Cauchy projections. The paper is of importance to very
      high-dimensional sparse approximations of the chi-square, which could find
      its usage in a lot of text and image applications. One main
      concern is that the writing is very bad, almost atrociously so. From the
      beginning it is extremely unclear what is the goal of the paper. The first
      theorem is almost irrelevant with the rest, as it gives a bound, just to
      be said not tight. And starting from section 4, the focus has turned to
      the alpha=1 case where links between the collision probability and the
      chi-square is proposed, and later proved in the binary case. The logic
      seems completely reversed and free-flowing, with occasionally theorems
      that don't lead to anywhere (e.g. Theorem 1, and the approximation (12)).
      The claims in the conclusion look out-or-order and random (e.g. lines
      424-427). In general, this looks more like a research memo rather than a
      rigorous, well-organized paper. Therefore I'm torn on a score between 5
      and 6 since it looks like this paper should be completely re-written
      before presented to readers. However, the finding itself is still
      quite interesting. Lemma 5 is a strong result that established a low error
      bound on the approximation in binary case. The result that one can
      approximate cos^{-1}(chi^2) by inner product is new. A less satisfying
      result is the way chi^2 is approximated, where cos needs to be applied to
      transform sign(x)* sign(y) so that the approximation is no longer an inner
      product thus don't enjoy computational savings in large-scale learning. It
      will be interesting to see whether the approximated cos^{-1}(chi^2) can
      translate to a linear approximation to the more useful chi2 CDF function,
      or whether some more theoretical and empirical results can be found for
      the cos^{-1}(chi^2). Other notes and relevant references: Note
      that when alpha = 1, the bound in Theorem 1 corresponds to the geodesic
      distance on the simplex, as defined in Lafferty and Lebanon.
      Diffusion Kernels on Statistical Manifolds. JMLR 2005. Taking the
      square root of all the input u, v is known as the Hellinger distance,
      widely used in both NLP and vision but widely known as inferior to
      chi-square. Chi-square is very important at least in computer
      vision, quite a few papers have worked on its approximation. There are
      some additional references that might be interesting for this topic:
      Sreekanth, Vedaldi and Zisserman. Generalized RBF feature maps for
      efficient detection. BMVC 2010 established an approximation for
      exp-chi2, which has been empirically verified as much better than the chi2
      kernel in classification accuracy. Also note exp-chi2 resembles the CDF of
      the chi2 distribution, hence corresponds to the p-value of the chi2-test.
      Therefore it's more appropriate than the chi2 itself as a comparison
      metric. Also relevant is a recent arXiv paper provides a linear
      approximation to chi2 with geometric convergence rate: Li, Lebanon
      and Sminchisescu. A Linear Approximation to the chi2 Kernel with
      Geometric Convergence. http://arxiv.org/pdf/1206.4074v3.pdf
      which presents a formula to approximate each dimension of the chi2
      kernel with several dimensions with geometric convergence rate. One can in
      principle construct an approximation of chi2 on very high-dimensional
      sparse features by doing a standard Gaussian random projection from such
      an approximation. It would be interesting to compare these two approaches.
      ---------------------------------------------------------------------
      Comments on Rebuttal: The point the authors mentioned on
      the ability for applications in streaming data is correct and interesting,
      and give some merit to the otherwise unjustified acos(chi2) metric (since
      it's computable for streams, even if it's not a well-understood metric
      yet). Therefore I decide to change the score to 6. The main concerns are
      still 1) Lack of theoretical and/or practical insights of the acos(chi2)
      kernel. 2) Writing and organization of the paper. Performance
      comparisons with the arXiv paper would be an interesting side note, but
      that was not a factor in the score I gave with the initial review. The
      score came from the above 2 concerns.


Relevance Topic Model for Unstructured Social Group Activity Recognition
Unstructured social group activity recognition in web videos is a challenging task due to 1) the semantic gap between class labels and low-level visual features and 2) the lack of labeled training data. To tackle this problem, we propose a "relevance topic model" for jointly learning meaningful mid-level representations upon bagof-words (BoW) video representations and a classifier with sparse weights. In our approach, sparse Bayesian learning is incorporated into an undirected topic model (i.e., Replicated Softmax) to discover topics which are relevant to video classes and suitable for prediction. Rectified linear units are utilized to increase the expressive power of topics so as to explain better video data containing complex contents and make variational inference tractable for the proposed model. An efficient variational EM algorithm is presented for model parameter estimation and inference. Experimental results on the Unstructured Social Activity Attribute dataset show that our model achieves state of the art performance and outperforms other supervised topic model in terms of classification accuracy, particularly in the case of a very small number of labeled training videos.

General: The paper proposes a supervised (hybrid)
      topic model for unstructured activity recognition. The model is named:
      relevance topic model (RTM). The supervised part is the label of the
      classes of the training videos. RTM is an integration of sparse
      Bayesian learning and Replicated Softmax. The main concept is to jointly
      learn discriminative topics as mid-level video representations and
      discriminant function as a video classifier. RTM is composed of an
      undirected part to model the marginal distribution of video words and
      a directed part to model the conditional distribution of video classes
      given the latent topics. Also, the authors propose the parameter
      estimation and inference methods. The authors evaluate their algorithm
      in the Unstructured Social Activity Attribute (USAA) presenting
      quantitative results of RTM. They compare the method with the literature
      improving in all cases the activity classification accuracy. Also the
      authors show an interesting comparison of the correlation of topics of two
      different classes. The idea seems interesting. The usage of SIFT,
      STIP and MFCC which are very low level features mostly based on pixel
      representations. It can be interesting a small discussion on how more
      descriptive features (object detection) can be added to the model.
      Relevant work that could be added in the Bibliography, [1] Similar
      subject [2] interesting approach on structured activity discovery.
      [1] Social Role Discovery in Human Events Vignesh Ramananthan,
      Bangpeng Yao, and Li Fei-Fei IEEE Conference on Computer Vision and
      Pattern Recognition (CVPR). Portland, OR, USA. June 23-28, 2013 [2] J.
      Varadarajan, R. Emonet and J.-M. Odobez Int. Journal of Computer
      Vision (IJCV), Vol. 103, Num. 1, pages 100-126, May 2013. Quality:
      The quality of the paper is good. Clarity: The paper is
      clear, the problem is well defined. Originality: The
      originality is Medium. This is not the first approach that aims at
      building intermediate features for activity recognition. Nor the first one
      that uses some full or semi-supervision. Unsupervised methods such
      as [2] for structured scenarios have strong assumptions in the temporal
      structure of the words (observation vectors) but in unstructured
      activities as the ones target in this work temporal constraints can be
      relaxed. Minimal supervision in the learning is acceptable for a
      difficult task as activity recognition in wild videos. An interesting
      question would be how many is the minimal human labeled data that is
      required to classify wild videos decently. Significance: The
      topic is exciting research, I would say of big significance.


Learning and using language via recursive pragmatic reasoning about other agents
Language users are remarkably good at making inferences about speakers' intentions in context, and children learning their native language also display substantial skill in acquiring the meanings of unknown words. These two cases are deeply related: Language users invent new terms in conversation, and language learners learn the literal meanings of words based on their pragmatic inferences about how those words are used. While pragmatic inference and word learning have both been independently characterized in probabilistic terms, no current work unifies these two. We describe a model in which language learners assume that they jointly approximate a shared, external lexicon and reason recursively about the goals of others in using this lexicon. This model captures phenomena in word learning and pragmatic inference; it additionally leads to insights about the emergence of communicative systems in conversation and the mechanisms by which pragmatic inferences become incorporated into word meanings.

This paper puts forward a model of learning
      vocabulary, in the context of a hypothetical 'naming game'. A row of
      objects visible ton two players, the 'speaker' and the 'listener'. There
      are repeated rounds; in each round an object is indicated to the speaker,
      who must then say a word that will cause the listener to point to that
      object. This game can be (and has been) set up to be played
      between people, but this paper considers simulated play between learning
      algorithms. Both players start with prior beliefs about a lexicon;
      they mutually refine these beliefs during repeated play. The paper
      starts with a nice discussion of different ways in which to model this
      vocabulary learning problem. In my view this is the best part of the
      paper. The authors point out that there can be deep recursions if both
      players attempt to optimise their communication strategy in a game
      theoretic manner, and they make an interesting point that this deep
      recursion does not seem to allow vocabulary learning. To prevent such
      recursion, the authors introduce what seems a nice idea that both players
      believe that there is some correct conventional lexicon, of which they
      themselves have incomplete knowledge. This allows apparently simpler
      shallow reasoning about what the lexicon may be. There are then
      case studies of two learning problems: uninformative prior beliefs, but
      objects with different frequencies and words with different costs; and the
      case of specificity implicature, where the problem is to learn that 'some'
      is pragmatically used to mean some, though it can also mean all. I
      feel that this is potentially a rather nice line of research, but I also
      feel that this paper has some problems in its current form. First,
      there are rather few simulations; the results of a few individual
      simulations only are given, and there is no systematic empirical analysis
      under a range of conditions. How do these algorithms fare with slightly
      larger problems? I take the point that different amounts of
      experimentation with simulations are expected in different fields -- but
      the authors should realise that if they propose an algorithm, they should
      give sufficient simulations to give some empirical idea of how it behaves,
      and what its behaviour is on problems of different sizes. Traces of
      individual runs are not enough. Second, I was a little unclear as
      to what the prior distributions over lexicons are. In the case of Horn
      implicature, each word is assumed to match a single object, so the prior
      can be represented as a multinomial distribution over the objects. In the
      case of specificity implicature, the word some can mean some or all, which
      seems to be represented as a probability distribution across objects: what
      is the prior distribution in this case? Thanks to the authors for
      clarifying this point. Third, a rather simpler model would be for
      a learner to maintain a current assumed lexicon; if there are
      communication failures, then the learner may change it. The learner (and
      speaker?) may then repeatedly change their own lexicons, until they reach
      a state where the lexicons match, communication succeeds, and no further
      changes are necessary. This idea might turn out to need a Bayesian
      formulation -- but there could be simpler possibilities requiring less
      computation? As a suggestion, what about two-word phrases? Can you
      produce a model of learning nouns when there are adjectives also? You
      could have arrays of objects with two features, and you have one word
      describing one feature, and the other word describing the other feature.
      You could only say 'red' if there is only one red object, but you could
      say 'red car' if there is a red car, a green car, and a red lorry. Your
      present examples are rather small...


Variance Reduction for Stochastic Gradient Optimization
Stochastic gradient optimization is a class of widely used algorithms for training machine learning models. To optimize an objective, it uses the noisy gradient computed from the random data samples instead of the true gradient computed from the entire dataset. However, when the variance of the noisy gradient is large, the algorithm might spend much time bouncing around, leading to slower convergence and worse performance. In this paper, we develop a general approach of using control variate for variance reduction in stochastic gradient. Data statistics such as low-order moments (pre-computed or estimated online) is used to form the control variate. We demonstrate how to construct the control variate for two practical problems using stochastic gradient optimization. One is convex-the MAP estimation for logistic regression, and the other is non-convex-stochastic variational inference for latent Dirichlet allocation. On both problems, our approach shows faster convergence and better performance than the classical approach.

quality: 5 (out of 10) clarity: 6 originality:
      6 significance: 9 SUMMARY: The authors propose to accelerate
      the stochastic gradient optimization algorithm by reducing the variance of
      the noisy gradient estimate by using the 'control variate' trick (a
      standard variance reduction technique for Monte Carlo simulations,
      explained in [3] for example). The control variate is a vector which
      hopefully has high correlation with the noisy gradient but for which the
      expectation is easier to compute. Standard convergence rates for
      stochastic gradient optimization depend on the variance of the gradient
      estimates, and thus a variance reduction technique should yield an
      acceleration of convergence. The authors give examples of control variates
      by using Taylor approximations of the gradient estimate for the
      optimization problem arising in regularized logistic regression as well as
      for MAP estimation for the latent Dirichlet Allocation (LDA) model. They
      compare constant step-size SGD with and without variance reduction for
      logistic regression on the covtype dataset, claiming that the variance
      reduction allows to use bigger step-sizes without having the problem of
      high variance and thus yields faster empirical convergence. For LDA, they
      compare the adaptive step-size version of the stochastic optimization
      method of [10] with and without variance reduction, showing a faster
      convergence on the held-out test log-likelihood on three large corpora.
      EVALUATION: Pros: - I like the general idea of variance
      reduction for SGD using control variates -- it could have a big impact
      given the popularity of SGD. - The motivation is compelling; the
      concrete examples of control variates are convincing; and the the general
      idea (Taylor approximation to define them) seems generalizable - The
      paper is fairly easy to read. Cons: - The experiments are
      somewhat weak: only one dataset for logistic regression; and a lack of
      standardized setup for LDA. - The related work is not covered.
      QUALITY: The theoretical motivation for the approach is compelling
      (reducing the variance of the gradient estimates reduces the constant in
      the convergence rate), but the execution in the empirical section is
      fairly weak. 1) For logistic regression, they only consider one
      dataset (covtype). As the previous SGD optimization literature has showed,
      there can be significant variations of behavior between different datasets
      (and step-sizes) -- see for example figure 1 and 2 of "A Stochastic
      Gradient Method with an Exponential Convergence Rate for Finite Training
      Sets", in the arXiv:1202.6258v4 version which compare SGD with different
      methods on covtype and other datasets for regularized logistic regression.
      2) For the LDA experiments, I am puzzled why the authors didn't re-use
      exactly the same setup as in [10] and [4] so that their results could be
      comparable? Why using a different mini-batch size (500 vs. 100); a
      different held-out test set (2k vs. 10k), etc.? It is suspicious that the
      results of their baseline on the held-out test set (which is the
      state-of-the-art method in 10] are all systematically worse than what was
      presented in [10]. The authors should clarify this in their rebuttal.
      3) Another compelling experimental baseline which is missing is to
      compare using different mini-batch sizes (which is also a variance
      reduction technique mentioned in the introduction) vs. their control
      variate method -- as shown in [4], the CLARITY: The paper is
      fairly easy to read. I like figure 1. An important point which needs to be
      clarified is how do they estimate the covariance quantities in their
      experiments to compute a* (do they use the empirical covariances on the
      mini-batch? This should be repeated in the experiments section -- and
      perhaps a discussion of how its cost compare to the standard SGD method
      should be included). Figure 2 is very hard to read -- the authors should
      add *markers* to identify the different lines. See also below for more
      suggestions. ORIGINALITY: I am not familiar of any work using such
      variance reduction techniques for SGD in such generality. On the other
      hand, the paper is lacking a coverage of related work. An important
      missing reference is the paper "Variational Bayesian Inference with
      Stochastic Search" by John Paisley, David Blei, Michael Jordan ICML 2012
      which also uses a control variate method to reduce the variance of a
      stochastic optimization approach to variational mean-field to do
      variational Bayes (they consider both Bayesian logistic regression and
      HDPs). The authors should explain in their rebuttal what novel
      contributions they make in comparison to this prior work. Another relevant
      piece of work (which is a useful pointer to mention though it doesn't
      compete with the novelty of this submission) is "Variance Reduction
      Techniques for Gradient Estimates in Reinforcement Learning" by Evan
      Greensmith, Peter L. Bartlett, Jonathan Baxter, JMLR 2004.
      SIGNIFICANCE: In addition to the two applications mentioned in
      this paper, the approach presented could be most probably generalized to
      many other settings where SGD has been used. Given the popularity of SGD
      for large-scale optimization, the potential impact of this work is quite
      significant. The empirical evidence presented is somewhat weak, but the
      theoretical intuition is fairly compelling and I could believe that a more
      thorough empirical comparison could also show significant improvements. I
      note that their theoretical argument didn't depend on having a finite
      training set; it would thus be interesting to see this approach used as
      well in the real stochastic optimization setting (where the full
      expectation cannot be computed) and where running averages are used to
      estimate the quantities. == Other detailed comments ==
      line 058: 'discuss' - > discussion line 089: The
      authors should be clearer that the matrix A *depends on w* as well.
      equation (5): Cov(g, h) is not symmetric in general -- so the 2nd
      term should be -(cov(g,h) + cov(g,h)^T). equation (6): it should
      be cov(h,g) on the RHS [or cov(g,h)^T], not cov(g,h) Paragraph
      150-154: It might be worthwhile to point out that in the maximal
      correlation case, one could set h_d = g_d and then the variance becomes
      zero (but obviously, we cannot compute E[h_d] efficiently in this case).
      lines 295-296: "This is different from the case in Eq. 11 for
      logistic regression, which is explicit." - > this sentence is
      ambiguous. What is explicit? figure 3: For which w did they
      compute the Pearson's coefficient? Is this using the true covariances or
      the estimated covariances from the mini-batch? === Update after
      rebuttal == The authors should carefully cover the related work in
      their final version of the paper, as well as implement the other
      corrections mentioned above (I will double check!). I agree that they make
      a nice contribution over the work by [Paisley et al. 2012]; on the other
      hand, they should also be clear that [Paisely et al. 2012] were already
      using control variates to improve the *optimization* of their variational
      objective; just that perhaps they were not using it in such generality as
      in this submission. I still think that the experiments are on the weak
      side, which is why my recommendation for acceptance is not stronger.


Regularized M -estimators with nonconvexity: Statistical and algorithmic theory for local optima
We establish theoretical results concerning local optima of regularized Mestimators, where both loss and penalty functions are allowed to be nonconvex. Our results show that as long as the loss satisfies restricted strong convexity and the penalty satisfies suitable regularity conditions, any local optimum of the composite objective lies within statistical precision of the true parameter vector. Our theory covers a broad class of nonconvex objective functions, including corrected versions of the Lasso for errors-in-variables linear models and regression in generalized linear models using nonconvex regularizers such as SCAD and MCP. On the optimization side, we show that a simple adaptation of composite gradient descent may be used to compute a global optimum up to the statistical precision stat in log(1/ stat ) iterations, the fastest possible rate for any first-order method. We provide simulations to illustrate the sharpness of our theoretical predictions.

summary: This paper extends a previous
      line of work on analysis of the effectiveness of estimators based on
      non-convex optimization; in particular, it provides statistical guarantees
      for a class of separable non-convex functions: those that are a sum of a
      loss function that is non-convex but has RSC, and regularizers which are
      potentially non-convex themselves. For such settings the paper shows the
      consistency of any local (as opposed to global, or special) minimum.
      clarity: The paper is clearly written, except for
      (a) the issue of the side-function g. Except for a brief
      discussion, there is not much to go by in trying to understand the purpose
      of this function. A better discussion of this would enhance the paper.
      Also maybe synthetic experiments to see whether this is needed in
      "practice". (b) it is casually mentioned that generalizations to
      other "decomposable" regularizers is possible. doing so would
      significantly increase the applicability of this work, e.g. extending to
      settings like low-rank matrices etc. originality: The
      reviewer feels the paper has limited originality. In particular, a lot of
      the proof borrows from the previous work of Loh and Wainwright; this fact
      is highlighted, for example, by the fact that the result holds for very
      specific regularizers: those which are separable into a sum of terms, one
      for each coordinate (like the \ell_1 norm is). Similarly, another
      generalization - to non-convex functions with RSC, instead of functions
      that arise from "corrected LASSO" in Loh-Wainwright - is not fleshed out
      much. Do these represent a straightforward generalization (along the lines
      of Negahban et al), or a substantial new advance ? significance:
      The significance of this paper is not immediately clear to the
      reviewer. In particular, while it is clear that a result about all local
      minima is more powerful than just a result for global minima or special
      initializations, it is not clear what are the settings where all the
      assumptions would hold. For example, the paper seems to be focused
      on sparse recovery type settings, given that the regularizer is separable
      and has other "ell_1 - like" properties. The authors give two examples
      which satisfy the assumptions, but these are not widely popular in the
      literature. What about, for example, the more popular example of \rho(t) =
      |t|^p for some p less than 1 ? Also, are there interesting/natural
      loss functions that are non-convex, but still have RSC and to whom the
      results of this a paper can be applied ? The paper seem to give no
      examples in this direction.


Robust Spatial Filtering with Beta Divergence
The efficiency of Brain-Computer Interfaces (BCI) largely depends upon a reliable extraction of informative features from the high-dimensional EEG signal. A crucial step in this protocol is the computation of spatial filters. The Common Spatial Patterns (CSP) algorithm computes filters that maximize the difference in band power between two conditions, thus it is tailored to extract the relevant information in motor imagery experiments. However, CSP is highly sensitive to artifacts in the EEG data, i.e. few outliers may alter the estimate drastically and decrease classification performance. Inspired by concepts from the field of information geometry we propose a novel approach for robustifying CSP. More precisely, we formulate CSP as a divergence maximization problem and utilize the property of a particular type of divergence, namely beta divergence, for robustifying the estimation of spatial filters in the presence of artifacts in the data. We demonstrate the usefulness of our method on toy data and on EEG recordings from 80 subjects.

This paper shows that the CSP algorithm (a very common
      algorithm for spatial filtering EEG data to increase the discriminability
      between classes characterized by differences in power/variance as in motor
      imagery) can be understood as a divergence maximization algorithm. They
      then use that insight to develop a CSP algorithm that is more robust to
      outliers by using the beta divergence. Clarity The paper
      is very clearly written. Figure 1 is a very nice summary of the early
      parts of the paper. Quality This paper is a very nice
      example of a theoretical contribution that ends up having practical
      application to real data. One minor concern I had was that I am not sure
      how realistic adding independent noise to each electrode is as many EEG
      artifacts are visible across multiple electrodes (e.g. blinks, muscle
      movements, EM noise, physical movements), however the authors do show an
      example of exactly this type of artifact in their real data. It would be
      nice to see how the algorithm performs on other types of artifacts and
      non-stationarities. Significance Improving the CSP
      algorithm could have a big impact on motor-imagery and steady-state evoked
      potential BCIs. Originality The authors mention that the
      proof that CSPs maximize divergence has been proved for the special case
      of 1 CSP filter. The rest of the work appears to be
      novel.


Which Space Partitioning Tree to Use for Search?
We consider the task of nearest-neighbor search with the class of binary-spacepartitioning trees, which includes kd-trees, principal axis trees and random projection trees, and try to rigorously answer the question "which tree to use for nearestneighbor search?" To this end, we present the theoretical results which imply that trees with better vector quantization performance have better search performance guarantees. We also explore another factor affecting the search performancemargins of the partitions in these trees. We demonstrate, both theoretically and empirically, that large margin partitions can improve tree search performance.

The paper presents bounds on the search performance of
      a simple, tree-based nearest neighbor search algorithm. The bounds
      depend on the vector quantization performance on the tree. It is
      argued that this result implies that trees with good vector
      quantization performance are advantageous for nearest neighbor search.
      The statement is extended to large margin splits. The title of
      the paper asks "which space partitioning tree to use for search"? It
      should better ask "which tree results in the strongest performance
      guarantees"? The paper says almost nothing about practical
      performance. This is mostly due to the choice of an artificially
      simplified search procedure. More often than not, a better guarantee
      is an artifact of a certain flavor of analysis or a proof technique,
      since we are only talking about upper bounds. If the bounds are not
      tight then the bounds say little about which tree to "use" (in
      practice!). This paper makes the common mistake of confusing a better
      performance guarantee with a guarantee for better performance. This
      happens at several spots, e.g., first sentence of section 4.
      Algorithm 1 is analyzed in depth. However, I am unsure how
      relevant this algorithm is. It descends the tree without backtracking.
      At the target depth l it performs exhaustive search. Although this is
      not taken into account in the analysis, the final search can be
      performed with efficient exact tree search, so, this time with
      backtracking. This algorithm does not find the exact nearest
      neighbor. The obvious alternative is to apply a heuristic for skipping
      some of the branches on the fly. The decisive difference is that in
      Algorithm 1 the decision which branches to traverse is not made in a
      data dependent manner, but instead based on a pre-specified parameter.
      This is why personally I would never use this algorithm. Since all
      results are restricted to this algorithm I question the relevance of
      this paper. I see that analyzing the computational complexity of a
      method with backtracking is surely much harder. I argue that doing so
      would be a prerequisite for understanding the behavior of realistic
      algorithms. I cannot get rid of the impression that this analysis was
      conducted simply because it is possible, and not because it is
      relevant. The logic of the conclusion is as follows: Algorithm
      1: good VQ performance => good search performance. Now Algorithm 1
      is not a good search algorithm by itself. When using more elaborate
      tree search procedures there remains little to nothing to conclude
      from the present analysis. However, the title as well as other
      statements in the paper (e.g., top of page 5) indicate that the
      conclusion is rather general. I want to emphasize that this is not the
      case, and thus this paper does NOT answer the question which search
      tree to use for search in practice. I would appreciate the
      result if it could help the analysis of more realistic search
      procedures. However, I am not an expert on the analysis of tree search
      and thus I cannot judge the present paper from this perspective. Also,
      this paper does not claim to provide new methods for analysis, it is
      all about the theorems. And this makes me question its value.
      The empirical evaluation is weak. Only four data sets are used,
      and they are even non-standard. E.g., why is MNIST sub-sampled to 6000
      training and 1000 test samples? This is arbitrary, an no reason is
      given. This does not help my trust in the evaluation. With low numbers
      of training points there is no real need for tree search at all. I
      see that the empirical results are nicely in line with the analysis.
      However, how about computation time? E.g., a single decision in a
      kd-tree is cheaper than in a PA-tree by a factor of O(dim(X)). The
      added computation time could be used for backtracking, which could
      well give the kd-tree an advantage. So once more, this analysis says
      nothing about which tree to use for search with a better algorithm.


Projecting Ising Model Parameters for Fast Mixing
Inference in general Ising models is difficult, due to high treewidth making treebased algorithms intractable. Moreover, when interactions are strong, Gibbs sampling may take exponential time to converge to the stationary distribution. We present an algorithm to project Ising model parameters onto a parameter set that is guaranteed to be fast mixing, under several divergences. We find that Gibbs sampling using the projected parameters is more accurate than with the original parameters when interaction strengths are strong and when limited time is available for sampling.

Aims to improve the mixing rate of Gibbs sampling in
      pairwise Ising models with strong interactions, which are known to be
      "slow-mixing". Several projections to "fast-mixing" models are proposed;
      essentially, parameters are identified which are as close as possible to
      the original model, but weak enough to satisfy a spectral bound
      establishing rapid mixing. Experiments show some regimes where this leads
      to improved marginal estimates for a given computation time, compared to
      variational methods (mean field and belief propagation variants) and Gibbs
      sampling in the original model. The technical content builds
      heavily on prior results establishing conditions for rapid mixing [4,8].
      But I haven't seen the idea of projecting onto the nearest rapidly mixing
      model, as an approximate inference method, explored before. On the
      domain of "toy tiny Ising models", results offer some nice improvements
      over a good set of baselines (standard sampling and contemporary
      variational methods). My main concern is that I think there should be more
      than the usual amount of skepticism as to whether these results will scale
      to larger models. Experiments involve comparisons of mixing times of
      samplers on different models, and it is hard to judge how these will scale
      with problem size. Also there is a (from all appearances) computationally
      expensive projection step required to build the fast-mixing model, the
      cost of which seems not to be accounted for. Finally, it is not at all
      clear that generalization beyond binary states will be possible, since
      establishing convergence bounds more generally is far more challenging.
      CLARITY: In general the presentation is clear and reasonably
      accessible, given the technical content. But there are some places where
      reorganization and clarification is needed: * The paper refers several
      times to "standard results" and even states these in the form of a theorem
      (e.g. Theorem 6) without reference or proof sketch. Lemma 5 is left
      unproven, the reference proves for a special case of zero-field. *
      Discuss properties of the dependency matrix R. For instance, it does not
      appear to be tractable due to the maximization, please state whether this
      is the case. (I presume this is why lemma 5 is invoked.) * The second
      half of Sec. 4 is nearly impossible to follow. Before Theorem 7 the text
      references g, M, and Lambda before they are introduced. Then the statement
      of Theorem 4 includes notation that is not really explained. If this
      optimization is going to be discussed, more explanation is needed.
      (Perhaps less time could be spent restating results from [8] which are not
      really used.) * In Sec. 1, KL(q||p) is used for both directions of KL
      divergence when the notation needs to be shifted for one. In the first
      paragraph, q is used for the true distribution and p for the tractable
      approximation; this is the opposite of almost all related literature.
      EXPERIMENTS: It appears that the time comparison in Figure 2
      does *not* include the computation required for projection. A somewhat
      ambiguous statement to this effect appears in Sec 6.1 but is unclear;
      please clarify and if it is the case, show results with projection time
      included. As it stands, the proposed methods essentially get to use the
      output of a sophisticated variational optimization without penalty, which
      certainly makes the improvement over standard Gibbs less convincing.
      It was disappointing not to see experiments on larger models,
      given that Gibbs mixing times often depend on problem size. There are
      certainly options for running experiments in regimes where junction tree
      is intractable, like using a model where symmetries let the true marginals
      be computed, or taking the output of a very long sampling run as truth.
      I understand that KL(\theta||\psi) is intractable in general, but
      it would still be interesting to explore here as a potential "best case"
      for how sampling in an approximate model would perform. (Junction tree
      could be used for the toy models used in the submission.) Mean
      field is a degenerate case of the reverse KL projection, as the paper
      points out, yet there is a large difference between mean field error and
      the error from reverse KL projection. This deserves discussion.


Learning to Prune in Metric and Non-Metric Spaces
Our focus is on approximate nearest neighbor retrieval in metric and non-metric spaces. We employ a VP-tree and explore two simple yet effective learning-toprune approaches: density estimation through sampling and "stretching" of the triangle inequality. Both methods are evaluated using data sets with metric (Euclidean) and non-metric (KL-divergence and Itakura-Saito) distance functions. Conditions on spaces where the VP-tree is applicable are discussed. The VP-tree with a learned pruner is compared against the recently proposed state-of-the-art approaches: the bbtree, the multi-probe locality sensitive hashing (LSH), and permutation methods. Our method was competitive against state-of-the-art methods and, in most cases, was more efficient for the same rank approximation quality.

Summary: The paper presents a method that learns a
      pruning algorithm for a VP-tree, in non-metric spaces. The idea is to
      estimate the decision function of the approximate nearest neighbor search
      in the VP-tree by sampling, and approximating it with a piecewise linear
      function. The learning to prune method is validated for the search
      efficiency against relevant baselines for prunning, and outperforms them
      substantially when the intrinsic dimensionality of the data is small.
      Clarity: The paper is mostly clearly written but sometimes does
      not really go into explaining the implementation details and the choice of
      some parameters (for example, why choose K=100, m=7, rho=8 and the bucket
      size = 10^5? Line 185,227,315) Originality: Learning to
      approximate the approximate nearest neighbor classification on a VP-tree,
      to the extent of my knowledge, is the first work that 'learns to prune'
      Significance: Nearest neighbor method is a very fundamental topic
      in search or classification; thus this learning-to-prune method which
      approximates the nearest neighbor search with a non-linear function would
      be of some interest to a wide audience. However, the datasets
      chosen for validation for the experiments seem rather simple and have
      low-dimensionality, which are far from realistic. (What is the result on
      the RCV-256, and SIFT for L2?) Also, whether the proposed method can
      achieve the desired the speed-up is not well justified for the metric
      space, which limits its application. For fast search in the metric space,
      there are existing methods that utilize LSH and embeddings. One relevant
      paper is as follows: [31] P. Jain, B. Kulis, K. Grauman, Fast Image Search
      for Learned Metrics, CVPR 2008.


Learning Hidden Markov Models from Non-sequence Data via Tensor Decomposition
Learning dynamic models from observed data has been a central issue in many scientific studies or engineering tasks. The usual setting is that data are collected sequentially from trajectories of some dynamical system operation. In quite a few modern scientific modeling tasks, however, it turns out that reliable sequential data are rather difficult to gather, whereas out-of-order snapshots are much easier to obtain. Examples include the modeling of galaxies, chronic diseases such Alzheimer's, or certain biological processes. Existing methods for learning dynamic model from non-sequence data are mostly based on Expectation-Maximization, which involves non-convex optimization and is thus hard to analyze. Inspired by recent advances in spectral learning methods, we propose to study this problem from a different perspective: moment matching and spectral decomposition. Under that framework, we identify reasonable assumptions on the generative process of non-sequence data, and propose learning algorithms based on the tensor decomposition method [2] to provably recover firstorder Markov models and hidden Markov models. To the best of our knowledge, this is the first formal guarantee on learning from non-sequence data. Preliminary simulation results confirm our theoretical findings.

- In this paper, a spectral learning based algorithm
      for learning Markov Model and HMM in a non-sequential setting is derived.
      The paper deals with learning Markov models and hidden Markov models from
      sparse realizations obtained at random times. Proper proofs for
      empirical moments are given. Also, a sample complexity bound has been
      provided. The paper is generally well-written and understandable. The
      comments are as follows: -Learning a sequential model in a
      non-sequential setting is not a concept that everybody is familiar of. It
      may be beneficial to briefly review the existing methods (possibly maximum
      likelihood based). Author(s) may argue that this can’t be done because of
      space constraints. However, I believe that, it would be better to motivate
      the non-sequential sequential model learning rather than to reproduce the
      tensor decomposition algorithm of Anandkumar et al. I think that the
      tensor algorithm can be introduced only by giving the general idea of
      expressing the parameters as symmetric observable tensors. (equation 2)
      That is, author(s) can exclude the algorithm 2 and theorem 1. -The
      proper moment equations for learning a Markov model in a non-sequential
      setting are defined in page 5. Using these moment equations, it possible
      to recover the expected transition probability matrix T, and parameters
      \pi. of the Dirichlet prior, using the symmetric orthogonal tensor
      decomposition algorithm of Anandkumar et al. from 2012. However, in order
      to recover the transition matrix itself (P), author(s) propose a search
      heuristic. In my opinion, this search algorithm can be made clearer to the
      reader by writing it as a pseudo-code. The assumption of presence of a
      zero entry in P makes sense if the number of states in the Markov chain is
      large, which may induce sparsity. Minor comment: isn’t \pi = \alpha /
      \alpha_0? Instead of using the proportional (at the beggining of page 5),
      it is better to give the exact equality to make the reader understand the
      moment equation proofs more easily. -Experimental results are
      limited to synthetic data. Since learning a sequential model in a
      non-sequential setting is the primal motivation of this paper, I think it
      is essential to validate the algorithm on a real life data. Moreover, a
      performance comparison with a more conventional learning algorithm is
      vital for the justification of the algorithm. -In section 4, it is
      indicated that as the number of data items N increases, the take off point
      in Figure1(a) gets closer to the true value r=0.3. I personally can not
      understand why the projection error is not zero when r is less than 0.3
      and it is zero when r is larger. Shouldn’t it be different from zero
      whenever r is not equal to 0.3? A comment on this issue would be helpful.
      Minor comment: On legend of figure 1, shouldn’t the logarithms be of
      base 10, instead of 2? -The general motivation behind choosing
      spectral learning algorithms for learning latent variable models over ML
      based approaches is also due to their speed. There is no mention regarding
      the proposed algorithm on this aspect. A speed comparison with a ML based
      approach (possibly EM) would be beneficial. It is also possible to use
      matrix eigendecomposition based algorithm of Anandkumar et al. (A method
      of moments for mixture models and HMMs, 2012) which would be
      computationally cheaper than the tensor decomposition approach.


A Kernel Test for Three-Variable Interactions
We introduce kernel nonparametric tests for Lancaster three-variable interaction and for total independence, using embeddings of signed measures into a reproducing kernel Hilbert space. The resulting test statistics are straightforward to compute, and are used in powerful interaction tests, which are consistent against all alternatives for a large family of reproducing kernels. We show the Lancaster test to be sensitive to cases where two independent causes individually have weak influence on a third dependent variable, but their combined effect has a strong influence. This makes the Lancaster test especially suited to finding structure in directed graphical models, where it outperforms competing nonparametric tests in detecting such V-structures.

This paper provides a very interesting nonparametric
      approach to test for three-variable interactions (and independence) using
      embedding properties of ISPD kernels. The paper is overall well-written,
      clear and exhaustive in its coverage of the underlying algebraic proofs.
      With the need for efficient statistical testing tools to detect complex
      interaction effects in bioinformatics, this work seems promising in its
      applications. Given the stated illustrative purpose of identifying
      gene interactions, it could have been nice to see experiments on
      non-synthetic data and/or comparison of performances to other genomics
      methods (such as based on partial correlation). Also, the possibility
      raised in the concluding paragraph that the test could be extended to more
      than 3 variables seems to be somewhat contradicted by the point raised in
      section 4.3. Finally, it would be of great interest to see how the method
      behaves in large-dimensional spaces. Minor issue: There is no
      reference to Figure 3 in the main text


A* Lasso for Learning a Sparse Bayesian Network Structure for Continuous Variables
We address the problem of learning a sparse Bayesian network structure for continuous variables in a high-dimensional space. The constraint that the estimated Bayesian network structure must be a directed acyclic graph (DAG) makes the problem challenging because of the huge search space of network structures. Most previous methods were based on a two-stage approach that prunes the search space in the first stage and then searches for a network structure satisfying the DAG constraint in the second stage. Although this approach is effective in a lowdimensional setting, it is difficult to ensure that the correct network structure is not pruned in the first stage in a high-dimensional setting. In this paper, we propose a single-stage method, called A* lasso, that recovers the optimal sparse Bayesian network structure by solving a single optimization problem with A* search algorithm that uses lasso in its scoring system. Our approach substantially improves the computational efficiency of the well-known exact methods based on dynamic programming. We also present a heuristic scheme that further improves the efficiency of A* lasso without significantly compromising the quality of solutions. We demonstrate our approach on data simulated from benchmark Bayesian networks and real data.

This paper introduces a method for finding Bayesian
      networks for continuous variables in high-dimensional spaces. The paper
      assumes a Gaussian distribution of any particular random variable when
      conditioned on its parent nodes. A LASSO objective function is used to
      construct a sparse set of parent nodes for each random variable, subject
      to an additional constraint that the resulting structure be an acyclic
      graph. The network structure constraint is framed as an ordering problem,
      and an A* search algorithm is proposed which finds a directed acyclic
      graph which maximizes the LASSO objective function. The LASSO objective
      function, minus the DAG constraint, is used as an admissible heuristic in
      the A* search. This search is still not fast enough for graphs with large
      numbers of nodes, so a method for further pruning the search space is
      introduced. The paper is written clearly, with clear motivation
      and sufficient detail for implementation by a third party. The use
      of the LassoScore for optimization in the dynamic programming algorithm,
      and using the unconstrained LassoScore as a heuristic function, is clever.
      The simulation studies adequately show that the approximate version of
      their algorithm still greatly outperforms conventional greedy approaches,
      while exhibiting fast runtimes even on large networks of nodes. The
      performance graphs indicate that even substantial pruning the search graph
      represents only minimal losses relative to the optimal solution.
      While the paper describes the process as learning network
      structure for continuous variables, using a linear regression model with
      fixed noise error (section 2.1) for the conditional distributions would
      seem to suggest assuming a network in which each random variable is itself
      linear Gaussian. Offhand this appears to be a fairly strong assumption;
      however, as noted in the rebuttal, recovering structure is a difficult
      problem even in Gaussian networks. The application described in
      section 3.1 is not very clear, and it is not described how stock price
      data is represented as a Bayesian network. This reviewer is not familiar
      with the use of Bayesian networks for stock market analysis, and it is not
      immediately clear to the uninitiated what sort of model is used, or why
      stock market data would exhibit a particular network structure. This is
      addressed by references in the rebuttal, which should be added to section
      3.1.


Zero-Shot Learning Through Cross-Modal Transfer
This work introduces a model that can recognize objects in images even if no training data is available for the object class. The only necessary knowledge about unseen visual categories comes from unsupervised text corpora. Unlike previous zero-shot learning models, which can only differentiate between unseen classes, our model can operate on a mixture of seen and unseen classes, simultaneously obtaining state of the art performance on classes with thousands of training images and reasonable performance on unseen classes. This is achieved by seeing the distributions of words in texts as a semantic space for understanding what objects look like. Our deep learning model does not require any manually defined semantic or visual features for either words or images. Images are mapped to be close to semantic word vectors corresponding to their classes, and the resulting image embeddings can be used to distinguish whether an image is of a seen or unseen class. We then use novelty detection methods to differentiate unseen classes from seen classes. We demonstrate two novelty detection strategies; the first gives high accuracy on unseen classes, while the second is conservative in its prediction of novelty and keeps the seen classes' accuracy high.

This paper proposes a model for labeling images with
      classes for which no example appear in the training set, which is based on
      a combination of word and image embeddings and novelty detection. Using
      distances in th embedding space between test images and unseen and seen
      class labels, the approach is able to assign a probability for a new image
      to be from an unseen class. This is later used to decide which classifier
      to use (one designed for seen classes the other for unknown ones). Results
      on CIFAR10 are provided. Clarity: The paper is clearly
      written. Its motivation and studied task are interesting and ambitious.
      Originality: There is not much technical novelty since image and
      word embedding methods come from previous work and so do novelty
      detections models (with a slight modification of [Kriegel et al. CIKM09]).
      Still, the combination of those approach appears to be brand new and is
      not trivial. Significance: The importance of the problem
      considered here make that this paper could have quite an impact. However,
      the true application setting of such work (large numbers of classes,
      imbalance) is still quite far from what is proposed in this paper, which
      is yet an interesting step towards it. Indeed, all experiments here
      considers balanced classes, and it seems that such embedding-based work
      might suffer with imbalance (and the isometric Gaussian assumption),
      especially when the majority are examples from unseen classes. Similarly,
      this system needs "well-chosen" unseen classes to perform well (as shown
      in Fig. 3, where the system performs as chance when unseen classes are cat
      and dog). Still, these difficulties are shared by any existing system that
      I know of. Quality: The overall quality of the work is good but
      some questions remain. The more important point is in Section 6.3, where
      the full pipeline is used. This shows that in this setting, the Gaussian
      model outperforms the more sophisticated LoOP approach. Could it be due to
      the cutoff fraction for log probabilities, which is well tuned for the
      Gaussian model? How has it been chosen? On the training set? The LoOP does
      not require (nor benefit from) such tuning. What is the proportion of test
      examples set as unseen and seen in this combined setting? In
      Section 6.6, it is not specified if LoOP or the Gaussian model was used.
      According to the shape of curve of Fig. 5, I would say the Gaussian model.
      Is it true? Does it mean that this simpler approach is actually the one to
      choose for good performances with increased number of classes?
      Other remarks/questions: - Are the word vectors $w_y$ updated?
      - There seems to be an error in the equation defining lof(.): in the
      expectation (denominator), shouldn't we read pdist_\lambda(q,C(q)) ? (q
      instead of f). - A legend should be added for Figure 4 (left and
      center) indicating which curve correspond to accuracy on seen class, and
      to unseen. - Comparison with (Lampert et al., 09) in Section 6.3 is
      interesting. One should also compare with "What Helps Where -- And Why?
      Semantic Relatedness for Knowledge Transfer" by Rohrbach et al. CVPR 2010,
      which improved over Lampert and proposes an approach that does not rely on
      attributes for zero-shot learning. - The results with random
      distractor noise should be added in Figure 6. After rebuttal: I
      read the rebuttal which answered conveniently some questions I had. But
      does not completely justify the novelty.


Hierarchical Modular Optimization of Convolutional Networks Achieves Representations Similar to Macaque IT and Human Ventral Stream
Humans recognize visually-presented objects rapidly and accurately. To understand this ability, we seek to construct models of the ventral stream, the series of cortical areas thought to subserve object recognition. One tool to assess the quality of a model of the ventral stream is the Representational Dissimilarity Matrix (RDM), which uses a set of visual stimuli and measures the distances produced in either the brain (i.e. fMRI voxel responses, neural firing rates) or in models (features). Previous work has shown that all known models of the ventral stream fail to capture the RDM pattern observed in either IT cortex, the highest ventral area, or in the human ventral stream. In this work, we construct models of the ventral stream using a novel optimization procedure for category-level object recognition problems, and produce RDMs resembling both macaque IT and human ventral stream. The model, while novel in the optimization procedure, further develops a long-standing functional hypothesis that the ventral visual stream is a hierarchically arranged series of processing stages optimized for visual object recognition.

Summary: This paper presents an algorithm for
      category-level object recognition that searches a large space of
      heterogeneous hierarchical neural network architectures using a boosting
      mechanism. The resulting network found via this algorithm is used to
      predict multiunit neural activity in monkey IT, as well as the similarity
      structure in monkey neural IT and human fMRI IT representations. This is
      the first reasonably successful prediction of neural activity in IT in
      response to complex naturalistic images of objects. Pros:
      Predicts neural activity in IT in response to complex naturalistic
      images of objects for the first time and with reasonable success. This is
      a significant achievement, and will be exciting to many at NIPS.
      Introduces a new algorithm for category-level object recognition,
      which optimizes a very complex, heterogenous neural network architecture
      using a boosting methodology. This will be of interest also to
      non-neuroscientists interested in deep learning and object classification.
      The comparisons to neural data are done to a high standard, with
      results replicated on three independently collected datasets (two monkey
      multiunit, one human fMRI). The paper is clearly written in
      general. Cons: The Krizhevsky et al. object recognition
      network should be included in the evaluation, since this model was shown
      to be promising in closely related prior work (e.g., [2]). This is a very
      natural baseline model to compare to. The Krizhevsky network was shown to
      achieve classification performance superior to that of Monkey IT, on the
      exact Neural Representation Benchmark dataset used in this paper.
      The paper describes running the proposed HMO algorithm to obtain a
      network N_{HMO} used in the comparisons to experimental data. It would be
      very interesting to learn more about the properties of this optimal
      network. What is its topology? Is the permitted heterogeneity in the space
      of networks used, or does this procedure end up using networks of some
      standard depth? Since intuition into what makes a good architecture is
      generally hard to come by, it would be useful to see this example. It
      would also be useful to describe other properties of the specific HMO
      implementation used in this paper, namely, how many networks were
      evaluated in the optimization of single stack networks, etc. As it
      stands it is not possible to determine whether the proposed algorithm is
      necessary to model IT well, or whether other state-of-the-art object
      recognition methods would perform comparably. This has bearing on the view
      advanced at the end of the discussion that the paper's results argue for
      one relatively fixed, unmodifiable "core recognition" network which feeds
      into more plastic "downstream specialists" responsible for learning
      different tasks. The Krizhevsky network, by contrast, would be plastic
      throughout all levels, with no obvious "two-step" arrangement. If it
      performs comparably in predicting IT responses, it would weaken the
      paper's argument in support of their dual system view. On the other hand,
      if the Krizhevsky network underperformed the proposed method, this would
      be very informative. As it stands, arguing in support of the two-step
      arrangement seems premature given the other tenable but unexamined models
      in the literature. Minor: The description of the HMO
      algorithm is hard to follow. The score equation in line 143, Sum N(F(s)),
      makes sense to me as E(N(s)). Also it appears that F is never defined. The
      reweighted score function of line 156 again makes use of the undefined F,
      F_1, etc, which presumably again is N. 229: "Matricx" ->
      "Matrix" Fig. 3A: include Y axis label


Fast Algorithms for Gaussian Noise Invariant Independent Component Analysis
The performance of standard algorithms for Independent Component Analysis quickly deteriorates under the addition of Gaussian noise. This is partially due to a common first step that typically consists of whitening, i.e., applying Principal Component Analysis (PCA) and rescaling the components to have identity covariance, which is not invariant under Gaussian noise. In our paper we develop the first practical algorithm for Independent Component Analysis that is provably invariant under Gaussian noise. The two main contributions of this work are as follows: 1. We develop and implement an efficient, Gaussian noise invariant decorrelation (quasi-orthogonalization) algorithm using Hessians of the cumulant functions. 2. We propose a very simple and efficient fixed-point GI-ICA (Gradient Iteration ICA) algorithm, which is compatible with quasi-orthogonalization, as well as with the usual PCA-based whitening in the noiseless case. The algorithm is based on a special form of gradient iteration (different from gradient descent). We provide an analysis of our algorithm demonstrating fast convergence following from the basic properties of cumulants. We also present a number of experimental comparisons with the existing methods, showing superior results on noisy data and very competitive performance in the noiseless case.

The authors propose new algorithms for determining
      independent components, when the data is corrupted by Gaussian noise. The
      results of alternative methods are affected by noise. The algorithm
      improves on the runtime of a method based on the 4th cumulant. The
      problem is relevant. Quality of the paper is high. The approach seems
      novel and the contributions significant. The experimental results
      are not totally convincing. In synthetic experiments most of the variants
      of ICA perform similarly. For a small number of samples, the
      quasi-orthogonalization based method performs significantly worse than all
      other methods. For a large number of samples, though, the method performs
      at least comparable, even in the noiseless setting. A benefit is
      demonstrated for a high Gaussian noise setting, if sufficient data is
      provided. It would be interesting how the different ICA methods
      compare in various real world applications, where the noise distribution
      is unknown a priori. Runtime experiments are provided only for the
      method at hand. To set these into context, the runtime of alternative
      methods should be included.


Online PCA for Contaminated Data
We consider the online Principal Component Analysis (PCA) where contaminated samples (containing outliers) are revealed sequentially to the Principal Components (PCs) estimator. Due to their sensitiveness to outliers, previous online PCA algorithms fail in this case and their results can be arbitrarily skewed by the outliers. Here we propose the online robust PCA algorithm, which is able to improve the PCs estimation upon an initial one steadily, even when faced with a constant fraction of outliers. We show that the final result of the proposed online RPCA has an acceptable degradation from the optimum. Actually, under mild conditions, online RPCA achieves the maximal robustness with a 50% breakdown point. Moreover, online RPCA is shown to be efficient for both storage and computation, since it need not re-explore the previous samples as in traditional robust PCA algorithms. This endows online RPCA with scalability for large scale data.

This paper considers the theoretical properties of
      online updating of a "robust" PCA. Unlike traditional RPCA the proposed
      method does not learn the usual decomposition of the data into low-rank
      and sparse components but instead outliers are handled by accepting
      observations into the estimate of the covariance matrix proportional to
      their variance along the previously estimated PCA directions and scaling
      the accepted observations by this probability. The proposed
      approach seems novel and certainly looks to be simpler than other
      approaches solving the same problem which are referred to in the paper.
      However it would have been nice to see a comparison of these methods.
      The technical contribution of the paper seems sound and
      illuminating however the important question remains: how likely is it that
      the initial condition on H(w^0) is satisfied. This also seems to limit the
      usefulness of the method in real applications since in the experiments the
      authors initialise with PCA computed with the known outliers removed --
      perhaps RPCA or similar could be used to get the initial estimate?
      It also seems like in high dimensions, there might be issues due
      to high dimensionality/low sample size when computing the initial
      estimates. There are a few other questions I am left with:
      What happens when the concept changes i.e. when the underlying data
      generating process changes? I have also noticed some possibly
      minor mistakes/typos. In section 3.1 some random variable \mu is
      mentioned but it is never defined and seemingly not mentioned again. Is
      this a typo? The sentence introducing \lambda is quite strange, it
      implies that \lambda has already been defined but in fact it has not.
      In several places "affect" is mixed up with "effect".
      Captions in Figs 2 and 3 are wrong. They should be number of
      batches. It would have been instructive to compare the results
      with batch mode RPCA and also report on timings to get an idea if there is
      a significant speedup.


Confidence Intervals and Hypothesis Testing for High-Dimensional Statistical Models
Fitting high-dimensional statistical models often requires the use of non-linear parameter estimation procedures. As a consequence, it is generally impossible to obtain an exact characterization of the probability distribution of the parameter estimates. This in turn implies that it is extremely challenging to quantify the uncertainty associated with a certain parameter estimate. Concretely, no commonly accepted procedure exists for computing classical measures of uncertainty and statistical significance as confidence intervals or p-values. We consider here a broad class of regression problems, and propose an efficient algorithm for constructing confidence intervals and p-values. The resulting confidence intervals have nearly optimal size. When testing for the null hypothesis that a certain parameter is vanishing, our method has nearly optimal power. Our approach is based on constructing a 'de-biased' version of regularized Mestimators. The new construction improves over recent work in the field in that it does not assume a special structure on the design matrix. Furthermore, proofs are remarkably simple. We test our method on a diabetes prediction problem.

CONTENT The paper proposes a new method to
      constructing confidence intervals (CI's) and to perform testing of
      individual parameters in high-dimensional problems of supervised
      learning under the sparsity assumption. Although the main focus is on
      the model of linear regression, an extension to more general cases
      is presented. A numerical illustration of the proposed methodology is
      carried out on the diabetes data. The main observation underlying
      this work is that in the high-dimensional Gaussian linear model, the
      LSE is inconsistent but unbiased and has a Gaussian distribution with
      a well-known covariance matrix. Thus it can be used for constructing
      CI's. On the other hand, the Lasso is consistent under suitable
      assumptions but has a relatively large bias and, therefore, cannot
      be used for constructing CI's. The solution, that has been already
      evoked in previous work by Javanmard and Montanari [9] and van de
      Geer et al. [23], is to construct CI's based on an estimator which is
      between the LSE and the Lasso, which has a bias that is small as
      compared to the variance, under suitable assumptions. The most
      important question here is which assumptions may be considered as
      ``acceptable'' in this context. Basically, one can simply use the LSE
      \hat\theta = pinv(X^T*X)*X^T*Y, where pinv(M) is the pseudo-inverse of
      a matrix M, and construct a CI based on it. The length of the
      corresponding CI for the j-th parameter will be
      \sigma*sqrt(pinv(X^T*X)_{jj}). Under some (very strong) assumptions,
      this value is of the order 1/sqrt(n) and everything is fine, even in
      high-dimensions. For instance, this can be done if X is an orthogonal
      matrix (for p=n). The proposal made by the authors combines
      the ideas of the aforementioned works with those of [Cai et al.,
      Estimating Sparse Precision Matrix: Optimal Rates of Convergence and
      Adaptive Estimation, 2012] (cf lemma 1). Recall that Javanmard and
      Montanari [9] and van de Geer et al. [23] proposed to take the
      ``intermediate'' estimator in the form hat\theta^u = Lasso + M * X^T *
      Residual(Lasso) where M is an estimator of the inverse covariance
      matrix Sigma of the design (one should take this matrix if it is
      available). An important remark made by the authors of the paper under
      review is that the only property required from M in order to make the
      theory work is the convergence to zero of |M*X^T*X - I|_infty.
      Furthermore, the length of the CI's being proportional to
      [M*X^T*M^T]_{ii}, the authors propose the smart idea of choosing M by
      solving a simple optimization problem that minimizes the largest
      length [M*X^T*M^T]_{ii} under the constraint that |M*X^T*X - I|_infty
      is bounded by a given constant. EVALUATION pros - the
      paper reads well - the mathematical results are correct - the idea
      for choosing M is original and, in my opinion, this choice is better
      than those proposed in previous works cons - it should be better
      explained why the proposed approach is better than the simple one
      based on the LSE. Just saying that X is rank deficient is not enough,
      since the pseudo-inverse of a rank deficient matrix may still have
      nonzero entries on the diagonal. - there are some claims in the
      introduction that over-evaluate the contributions as compared to the
      previous work. For instance, line 53, the authors claim that they use
      essentially no condition beyond the standard ones for consistency of
      the Lasso. This is not true, since among the assumptions they make, I read
      s_0=o(sqrt(n)/log(p)) [line 208] and norm(Sigma^{-1})_\inf =
      o(sqrt(n/log(p))), which are much stronger than those required for the
      Lasso to be consistent. Similarly, lines 128-129, the authors' claim
      that van de Geer et al. [23] assume that n=Omega(s_0^2\log p) can be
      often interpreted as the results of the paper under review do not need
      such a restrictive assumption (which is not true). - The
      assumption norm(Sigma^{-1})_\inf = o(sqrt(n/log(p))) seems quite strong.
      Some discussion of it is necessary. - The choice of the parameter
      gamma, which may be very hard since it will depend on several unknown
      parameters (such as the largest diagonal entries of Sigma and its
      inverse) is not discussed. - [minor, can be easily fixed during the
      revision] some important references are missing: * there are two
      papers on the CI's in the sparsity setting: - Gautier and Tsybakov
      [arxiv.org/pdf/1105.2454.pdf] - Nickl and van de Geer
      [arxiv.org/pdf/1209.1508] * for the scaled Lasso quoted on lines 220
      and 225, the earlier work by Belloni et al. (Biometrika 2011) should
      be cited. - [minor, can be easily fixed during the revision] Some
      mathematical claims lack rigor. Here is a non exhaustive list * line
      77: it should be stressed that W and X are independent * line 182: the
      authors define the meaning of ``w.h.p.'', by the convergence of the
      probability to zero as n\to\infty. However, in the high-dimensional
      setting, n is not the only quantity that tends to infinity. So the
      authors should make precise what happens with p and s_0. Are they fix
      or do they go infinity with n? I guess that the results of the paper
      require p to go to infinity. This should be stated explicitly early in
      the text. * line 199: I did not manage to find any clear bound on
      (M_*SigmaM_*^T)_{ii} in Section 3.2. Please be more precise in the
      reference. * line 201: I do not agree that Thm 2.1 does not make any
      assumption on theta_0. In fact, thm 2.1 says that X satisfies the
      compatibility assumption with S the support of theta_0. If this
      support has more than n elements, this may not be true. So thm 2.1
      contains a contraint on the support of theta_0. * Eq (6) is a
      catastrophe from a mathematical point of view in terms of writing.
      First, the left hand side is a random variable, while the right hand
      side is deterministic. Second, since everything depends on x, which
      varies in R, it is important to explain if o(1) is uniform in x or
      not.