Learning Multi-level Sparse Representations
Bilinear approximation of a matrix is a powerful paradigm of unsupervised learning. In some applications, however, there is a natural hierarchy of concepts that ought to be reflected in the unsupervised analysis. For example, in the neurosciences image sequence considered here, there are the semantic concepts of pixel → neuron → assembly that should find their counterpart in the unsupervised analysis. Driven by this concrete problem, we propose a decomposition of the matrix of observations into a product of more than two sparse matrices, with the rank decreasing from lower to higher levels. In contrast to prior work, we allow for both hierarchical and heterarchical relations of lower-level to higher-level concepts. In addition, we learn the nature of these relations rather than imposing them. Finally, we describe an optimization scheme that allows to optimize the decomposition over all levels jointly, rather than in a greedy level-by-level fashion. The proposed bilevel SHMF (sparse heterarchical matrix factorization) is the first formalism that allows to simultaneously interpret a calcium imaging sequence in terms of the constituent neurons, their membership in assemblies, and the time courses of both neurons and assemblies. Experiments show that the proposed model fully recovers the structure from difficult synthetic data designed to imitate the experimental data. More importantly, bilevel SHMF yields plausible interpretations of real-world Calcium imaging data.

The authors present an algorithm for learning
      multilevel sparse representations. This is modeled as a (penalized) low
      rank matrix factorization problem where the number of factors is typically
      greater than 2 and each factor represents a different entity in the
      hierarchy. The factors are estimated using standard block co-ordinate
      descent methods where each factor is optimized while the others are kept
      first and sparsity promoting penalties are used at each step. The
      method is discussed in the context of analyzing large scale calcium
      imaging data where the assumption is that the neurons can be collected
      into a set of (possibly overlapping) neuronal assemblies giving rise to
      the hierarchy pixel --> neuron --> assembly. The goal then is to
      estimate the location of each neuron (in terms of pixels), the membership
      of each neuronal assembly, and the spikes of each assembly. The
      authors argue that their method performs better than other methods in
      artificial (but realistic) data and that it can correctly identify neural
      assemblies in real data. Originality The ideas
      presented in this paper are novel to the best of my knowledge. The
      three-way decomposition approach is interesting and I liked the fact that
      the authors also keep a standard two-way decomposition in their model and
      enforce consistency between the two. This extra step can make the
      algorithm more robust to phenomena such as a neuron in an assembly
      "misses" a spike etc. Quality My main concern about this
      paper is efficiency. Large scale calcium imaging experiments typically
      involve tens of thousands of pixels and thousand of timesteps. Although
      the multilevel decomposition reduces significantly the dimensionality, and
      several steps of the algorithm can bee parallelized, I can imagine that
      the algorithm might still be quite slow and require a large number of
      iterations. These points should be discussed. I am also not
      convinced by the usage of neural assemblies. It is not clear that this is
      something that will often appear in practice. Also, if a subset of neurons
      have the same spikes then this will be reflected in the temporal component
      of the spatiotemporal matrix. The rank of the matrix will still be small
      and I don't see a direct benefit from incorporating the proposed
      multilevel hierarchical decomposition. The authors would probably need to
      expand their opinion on this issue. Moreover, the various
      regularizers \Omega_U, \Omega_A should be clarified. For example, what are
      the convex cell shape priors that are used? The authors leave to
      the user the choice of regularization weights. This seems an important
      task to me since the number of regularizers is large and their use of
      critical importance. The authors claim that an accurate prior estimation
      of the rank is not of great importance since the algorithm will shrink non
      relevant components to zero. Although this sounds intuitive I can imagine
      that this behavior depends heavily on the right choice of the various
      sparsity penalties that will shrink the non relevant components.
      Clarity I think the paper is not clearly written at
      all and right now is only accessible to an expert of calcium imaging data
      analysis. For example, the authors mention very briefly in the results
      section how the neural spikes are translated into calcium activity through
      a convolution operator and do not discuss this central aspect of spike
      deconvolution at all in their algorithm. In fact it seems to me that they
      sidestep this issue "by eliminating all transients that overlap more than
      20%" (if I understand correctly). This is a topic of active research (e.g.
      Vogelstein et al. 2010, Grewe et. al. 2010) and I'd like to see how the
      authors relate to that literature. I found the inclusion of the
      main equations as a figure a bit unusual, although that is a question of
      style as well. Finally, I would prefer a more thorough
      presentation of the real data experiments. Although videos cannot be
      presented easily in paper, the results as shown in Fig 6 could be more
      convincing. In general, Fig 6 needs a better
      explanation.


Distributed Representations of Words and Phrases and their Compositionality
The recently introduced continuous Skip-gram model is an efficient method for learning high-quality distributed vector representations that capture a large number of precise syntactic and semantic word relationships. In this paper we present several extensions that improve both the quality of the vectors and the training speed. By subsampling of the frequent words we obtain significant speedup and also learn more regular word representations. We also describe a simple alternative to the hierarchical softmax called negative sampling. An inherent limitation of word representations is their indifference to word order and their inability to represent idiomatic phrases. For example, the meanings of "Canada" and "Air" cannot be easily combined to obtain "Air Canada". Motivated by this example, we present a simple method for finding phrases in text, and show that learning good vector representations for millions of phrases is possible.

This paper has 3 main contributions: (i) it extends
      the skip-gram model of (Mikolov et al, 2013) to speed up training by
      adopting an objective close to Noise Contrastive Estimation and
      subsampling frequent words, (ii) it proposes to learn phrase
      representations and introduces a dataset to evaluate those, (iii) it
      introduces the concept of additive compositionality. This is a
      good paper. It is clear and reads well. It is technically correct. The
      part on speeding up learning (i) justifies the paper. The additive
      compositionality (iii) is a nice concept to throw in, even if the authors
      make no attempt to explain how it arises from the optimization problem
      that training is solving. The part on phrase representation (ii) could be
      given more emphasis given it could be more controversial. In particular,
      testing the limits of the proposed technique might make the paper more
      insightful. ** NCE & Subsampling ** I feel that you
      should spend more time introducing NCE and highlighting the difference
      with ICE. From the text, it seems that the notation in (4) is wrong: you
      sample k sample w according to Pn(w) then you use each individual sample
      log(sigma<,>) as an unbiased estimator of the expectation but you
      never manipulate the expectation itself. For subsampling frequent
      words, it might be worth mentioning that this strategy might be
      inefficient for less semantic NLP task (e.g. POS tagging or parsing) where
      the representation of common words might matter more than for the semantic
      test from [7]. ** Additive compositionality ** This
      property is similar to the one highlighted in [Tomas Mikolov, Wen-tau Yih,
      and Geoffrey Zweig, Linguistic Regularities in Continuous SpaceWord
      Representations] which observed [Koruna-Czech] \simeq [Yen-Japan]
      You make the remark that these two kind of vectors are also close to
      currency. This is an interesting observation. It would be nice if one
      would attempt to explain how the optimization problem solved by maximizing
      (4) actually yields to such property. This remains puzzling to me and
      maybe to some other member of the ML community. ** Phrase
      representation ** You propose to learn a representation for each
      frequent (according to (6)) phrase. This is a valid proposal and your "Air
      Canada" example against Socher like models make sense. I do not have a
      strong argument against your strategy when data are plentiful. However, a
      strong argument in favor of distributed LM as opposed to n-gram LMs that
      they allow parameter sharing across similar contexts. This help modeling
      language because there will always be infrequent phrases that can benefit
      from similar frequent phrases, e.g. learning that "the (small town name)
      warriors" is a sports team need to share parameters with other popular
      teams. It might worse testing the limit of your strategy in terms of
      number of occurrences as it might go a long way. ** Typos **
      - Table 3: samsampling -> subsampling - It might be worth
      citing Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig, Linguistic
      Regularities in Continuous SpaceWord Representations which first
      introduced the unigram fact dataset.


When in Doubt, SWAP: High-Dimensional Sparse Recovery from Correlated Measurements
We consider the problem of accurately estimating a high-dimensional sparse vector using a small number of linear measurements that are contaminated by noise. It is well known that standard computationally tractable sparse recovery algorithms, such as the Lasso, OMP, and their various extensions, perform poorly when the measurement matrix contains highly correlated columns. We develop a simple greedy algorithm, called SWAP, that iteratively swaps variables until a desired loss function cannot be decreased any further. SWAP is surprisingly effective in handling measurement matrices with high correlations. We prove that SWAP can easily be used as a wrapper around standard sparse recovery algorithms for improved performance. We theoretically quantify the statistical guarantees of SWAP and complement our analysis with numerical results on synthetic and real data.

This paper proposes a wrapper called SWAP that can be
      applied around a given sparse solution with the potential for improving
      the likelihood that the maximally sparse (L0 norm) solution can be found.
      The procedure involves simply optimally swapping pairs of basis vectors in
      the sense of minimizing a quadratic penalty term. Both theoretical and
      empirical evidence are provided indicating that this proposal is more
      robust to dictionary correlations, which may disrupt convex L1-norm-based
      algorithms like Lasso or greedy techniques such as OMP or CoSaMP. At a
      high level, the basic idea seems to be that, while most existing methods
      are sensitive to correlations among 2k or more dictionary columns (where k
      is the sparsity level of the L0 solution), the proposed SWAP algorithm is
      only sensitive to such correlations among 2k or fewer, a less stringent
      condition for guaranteeing successful sparse recovery. One of the
      primary advantages of SWAP is that it can be applied to potentially
      improve the estimation quality of a solution produced by virtually any
      sparse estimation technique. Moreover, under somewhat technical conditions
      which depend on the cardinality and support of the maximally sparse
      solution, it may conceivably recovery the true support even when other
      algorithms fail. Although concrete examples where this provably occurs are
      lacking (more on this below), in general I found the paper to be mostly
      well-composed and thought-provoking. A significant downside though
      is that the formal recovery results depend on deploying SWAP while
      actually knowing the cardinality k of the optimal solution, which is
      generally not known in practice, and it is unclear how the algorithm will
      degrade when k is not available in advance. This is unlike the Lasso,
      where provable recovery guarantees can be derived without such explicit
      knowledge. As a second potential limitation, while it may well be true
      that SWAP has not been formally proposed previously, there is likely a
      very practical reason for this. Namely, SWAP is not scalable to large
      problem sizes unlike other greedy selection methods like OMP. More
      concretely, in the realistic scenario when number of measurements n, basis
      vectors p, and the sparsity level k all increase linearly, the complexity
      of a single SWAP iteration would increase by at least the cube of the
      common scale factor. In contrast, other greedy algorithms scale no more
      than quadratically. Moreover, the convergence rate is still an open
      question. In all of the experiments provided, the sparsity k was very low,
      only k=20 or smaller. In such a restricted scenario, it is not surprising
      that a few basis functions swaps here and there can lead to considerable
      improvement relative to baseline, both in estimation quality (support
      recovery percentage) and convergence rate. It would be interesting however
      to see how SWAP works in a more realistic higher-dimensional setting.
      The setup for Theorem 4.1, the primary result, takes a bit of time
      to digest and could possibly be augmented by a more comprehensive,
      intuitive description beyond the few remarks provided. In particular, it
      would be helpful to give an example dictionary such that conditions (7)
      and (8) are satisfied, while the corresponding conditions for Lasso, etc.,
      are not. Note that in the experimental results section, a block diagonal
      dictionary is proposed; however, it would appear that with such a
      dictionary it is unlikely that (7) and possibly (8) would be satisfied in
      this case. Assuming d > 1 (meaning the initial solution given to SWAP
      has more than one inactive element per the definition in the paper), there
      will generally always be a subset S of k atoms such that all of the
      associated inactive elements in this subset will be correlated with an
      active element in the subset. This is true because any active atom has a
      group of highly correlated atoms by construction, most of which are not
      part of the active set. Consequently, unlike analogous recovery results
      pertaining to Lasso (which incidently do not require any knowledge of the
      true support pattern, only the value of k), it is much less intuitive
      exactly where the advantage of SWAP lies, and specific dictionary
      structures where Lasso provably fails and SWAP provably succeeds. In other
      words, it seems that the situation described on line 205 is frequently in
      fact true for correlated dictionaries, and hence some further comment is
      warranted. Additionally, because the paper is primary of a
      technical nature, with involved proof techniques of the flavor developed
      in various other statistical settings, e.g. references [5,8–10,12], it
      would be helpful from a reviewing standpoint to give a few words regarding
      which aspects of the proof most resemble existing work and which aspects
      are the most novel. While I investigated parts of the proof in the lengthy
      supplementary section, I admittedly did not check it all (note that on
      line 492 of the supplementary, I believe the first instance of "active"
      should be switched to "inactive"). Other comments: * I
      would be curious how well a slight, far more computationally tractable
      modification of SWAP might perform. Specifically, what if instead of
      searching for the best pair of atoms to swap, you instead first find the
      best atom to add, then find the best atom to delete, and iterate back and
      forth? At the very least, the complexity would decrease drastically,
      although it is not clear how the theoretical and empirical results might
      change. * Is there a mistake on line 221 regarding the relative
      vector sizes? It would appear generally that max(d+1,k) = k in all of the
      most interesting cases (basically except when the estimated support has no
      overlap at all with the true support, such that d = k). Perhaps the max is
      supposed to be a min? * Is there a factor of 16 missing on line
      269 of the paper? * Line 291 seems to imply that the number of
      measurements n need only scale logarithmically with k (and p-k). How is
      this possible, since doesn't this mean that n could be smaller than k at
      some point? Additionally, do these claims regarding the number of
      measurements assume that conditions (7) and (8) are somehow automatically
      satisfied asymptotically? Because it would seem that it would all depend
      on how a correlated dictionary is that is being expanded in high
      dimensions, which has not been formally defined in the paper unless I'm
      missing something obvious. * On line 310 it does it mean that only
      four active variables are assigned to the same block (and the rest
      randomly dispersed), or alternatively that, given k = 20, five different
      blocks each have four active variables?


Integrated Non-Factorized Variational Inference
We present a non-factorized variational method for full posterior inference in Bayesian hierarchical models, with the goal of capturing the posterior variable dependencies via efficient and possibly parallel computation. Our approach unifies the integrated nested Laplace approximation (INLA) under the variational framework. The proposed method is applicable in more challenging scenarios than typically assumed by INLA, such as Bayesian Lasso, which is characterized by the non-differentiability of the 1 norm arising from independent Laplace priors. We derive an upper bound for the Kullback-Leibler divergence, which yields a fast closed-form solution via decoupled optimization. Our method is a reliable analytic alternative to Markov chain Monte Carlo (MCMC), and it results in a tighter evidence lower bound than that of mean-field variational Bayes (VB) method.

Overview == The paper proposed a form of
      variational distribution where dependency is assumed in the approximation.
      We have q(x,\theta) = q(\theta)q(x|\theta), and q(\theta) is represented
      as a series of Dirac-deltas. The idea resembles INLA, but with the clear
      advantages of VB (i.e. a bound on the marginal likelihood.)
      Discussion == Introduction -- This is a good
      introduction which lays out the paper and makes me curious the read
      further. Section 2 -- The ideas seems like a good one, and
      you describe the method in some detail in this section. The problem boils
      down the the minimisation of the objective in (2). My concern here
      is the entropy of the 'gridded' parameter. You move from a continuous
      integral in (2) to a sum with Dirac deltas in (3): what happens to the
      entropy in q(\theta)? 2.3: I'm not an expert on INLA, so
      perhaps other reviewers can comment here. I like the idea of a comparison,
      but struggle a little with your motivation. You satae that INLA cannot
      cope with sever non-Gaussianity in the posterior: how do you propose that
      the VB method will work better? I understand that the approximation is
      different, and that the variational method is based on a well-defined
      metric (KL divergence), but I fail to see how it helps with Gaussianity.
      Perhaps just a change of language is needed? Section 3 --
      Here you lay out the application of the method to Bayesian Lasso. The
      idea is to marginalise the parameters \theta (which I guess are the noise
      variance \sigma^2 and the prior width \lambda) on a grid, and for each
      grid point compute the optimal distribution q(x|\theta). It's great that
      the optimiation of each conditional distribution is convex: could you
      comment on wht would happen if it were not? Would it be necessary to find
      the global optimimum each time, or would the method be robust to that?
      On the first paragraph of page 5, you describe how the method
      preceeds, but I'm still concerned about the entropy of q(\theta).
      3.2 and 3.3: The upper bound on the KL divergence is an
      interesting proposition. I guess it's specific to the Bayesian Lasso model
      since it relies on the triangle inequality in Lemma 3.1. I confess
      that I'm a bit bamboozled by the idea of a Baysesian Lasso. To me, the
      lasso is a shrinkage method which helps in selecting a sparse
      point-estimnate solution. The Bayesian posterior is necessarily not
      sparse. Perhaps I should read the Bayesian Lasso paper, but you might
      include some more description of the model? Section 4 --
      Very nice experiments, well presented results which make the
      advantages clear. It troubles me a little that these experiments are
      rather small though, it would have been nice to see it applied to bigger
      data sets. Pros == - A nice formal framework, with
      (theoretical) comparison to INLA - A relevant and interesting problem,
      of interest to large parts of the community - an interesting upper
      bound on the KL which gives a one-shot solution (or initialisation)
      Cons == - Application is presented only to the Bayesian
      Lasso - Experiments are well presented but a little limited
      Summary == An interesting idea which is of interest to
      large parts of the community, with just enough experimental evidence to
      convey its practicality. Errata == 160: integrat_ing_ out.


Stochastic blockmodel approximation of a graphon: Theory and consistent estimation
Non-parametric approaches for analyzing network data based on exchangeable graph models (ExGM) have recently gained interest. The key object that defines an ExGM is often referred to as a graphon. This non-parametric perspective on network modeling poses challenging questions on how to make inference on the graphon underlying observed network data. In this paper, we propose a computationally efficient procedure to estimate a graphon from a set of observed networks generated from it. This procedure is based on a stochastic blockmodel approximation (SBA) of the graphon. We show that, by approximating the graphon with a stochastic block model, the graphon can be consistently estimated, that is, the estimation error vanishes as the size of the graph approaches infinity.

The authors describe a new method for predicting a
      family of random graph generators (graphons which are completely described
      by a function w:[0,1]^2 -> [0,1]) from a series of observed graphs
      generated from the random graph generator. They develop a method that
      approximates the function w using step functions for the case where the
      true function w is piecewise Lipschitz. The paper takes a new
      approach to a previous method in Lloyd et al, avoiding the need to place a
      prior distribution over w. Instead their approach is to apply a clustering
      method to group similar vertices across all the observed graphs and use
      averages in these blocks to approximate w by a step function. They
      compare their new method against other methods for predicting the random
      graph generator and show significant improvement over a range of different
      scenarios. Quality ---------------- This paper has many
      spelling mistakes, typos and layout problems, including the title which I
      think should be perspective and not perpective. I suggest that the authors
      give this paper a couple of proof reads to eliminate the errors. The ones
      I found whilst reading this paper are In the paper: - The
      title - Sentence just above eqn 5 - bottom of pg 3 has the title
      of a subsection with no text below it - The last sentence before
      section 3 - First sentence at the top of pg 6 references an appendix I
      think the authors mean the supplementary material - Typo in eqn 10
      w(u_i_y,j_v) I think should be w(u_i_y,u_j_v) - Theorem 3 uses
      S\in\theta(n) and then subsequently the text refers to S_n. I think the
      authors mean S_n here since I believe it is meant to be a subset of S =
      \{1,....,n\} \ \{i,j\}. - Second sentence of sect 4.1.2 send -> end
      In the supplementary material - ?? in theorem 2 - Lemma 3
      mentions a proposition 8 and there is none - eqn 5 has a typo (see
      note about eqn 10 in list above) - The statement of Lemma 2 is
      grammatically incorrect - The statement of Lemma 3 is grammatically
      incorrect - Theorem 3 again states S but then uses S_n - p10
      bottom half mentions a lemma 8 and there is none (I think they mean lemma
      4) - p12 second eqn up from bottom d_ij is missing a superscript c
      - p12 last equation the third f_ij does not need a sup
      Clarity -------------------------- In general, typos
      and spelling mistakes aside, this paper is coherent and clear. There were,
      however, a few points that could be made clearer In the Paper:
      - The use of S and S_n was not clear and I found this confusing, I
      think that this is possibly a typo (see above) - The first experiment
      4.1.1 compares their method to others for an arbitrary graphon. To make
      the comparison fair the authors use graphs of size n for the other methods
      and two graphs of size n/2 for their method. The second plot showing how
      their method improves with increasing numbers of observed graphs then
      seems to use graphs of size n. This could be made clearer. - The
      discussion of graphs with missing links talks about applying a random
      binary matrix M to the observed graphs to eliminate some of the links. It
      was unclear if the M remained constant across all the observed graphs
      G_1...G_2T and then was changed for each set of observed graphs or if a
      new M was used for each of the G_1 .. G_2T. If it was a the first way
      round this would lead to a consistent bias in the observed graphs (since
      vertices would consistently have missing links throughout). I think the
      latter approach of randomising M for each graph would be more natural
      approach. In the supplementary material: - I felt that since
      lemma 1 - 4 were proved before theorem 3 it would make sense to place
      lemma 5 here also. Originality -------------------------------
      The paper seemed original but I have had limited exposure to this area
      of research and so can't be certain. Significance
      --------------------------------- The paper seemed to be
      significant, but again I have had a limited exposure to this area of
      research so I cannot be certain.


Robust Bloom Filters for Large Multilabel Classification Tasks
This paper presents an approach to multilabel classification (MLC) with a large number of labels. Our approach is a reduction to binary classification in which label sets are represented by low dimensional binary vectors. This representation follows the principle of Bloom filters, a space-efficient data structure originally designed for approximate membership testing. We show that a naive application of Bloom filters in MLC is not robust to individual binary classifiers' errors. We then present an approach that exploits a specific feature of real-world datasets when the number of labels is large: many labels (almost) never appear together. Our approach is provably robust, has sublinear training and inference complexity with respect to the number of labels, and compares favorably to state-of-the-art algorithms on two large scale multilabel datasets.

This is a well-written paper that makes an interesting
      contribution to an important topic. In fact, the idea of using Bloom
      filters for dimensionality reduction in multi-label classification is
      intriguing. This seems to be a companion paper of another paper, in which
      this idea has already been introduced. The contribution of this paper is
      to improve the encoding scheme used in the Bloom filter. To this end, the
      structure of the label space is exploited by means of clustering methods.
      As claimed by the authors, this encoding makes the filter more robust.
      While I'm essentially fine with the idea of the method and
      theoretical part of the paper, I am much less convinced of the empirical
      study. First, given the goal of efficiency, one would certainly expect an
      empirical analysis of runtime in the experimental part, not only
      predictive accuracy. Moreover, the two data sets used in the
      experiments are somewhat debatable, especially due to their extremely low
      label cardinality (it would be interesting to see the marginal label
      distribution). In addition to the sparsity, the distribution will probably
      be skewed, so that most of the labels will almost never occur. In fact,
      one may wonder whether such data sets are sufficiently representative to
      provide an idea of how the authors' approach performs. Indeed, the
      performance of this approach (like the performance of other compression
      schemes) strongly depends on the label distribution, which is very
      specific for the data sets used. Moreover, losses like Hamming,
      which the authors focus on, are debatable in such cases, especially since
      always predicting negative will already yield an extremely low loss. For
      example, the author report a Hamming loss of 0.000734 for SBF and "only"
      0.000705 for RBF. Is this tiny difference of any meaning? Hard to say,
      especially since no standard deviations are reported.


Visual Concept Learning: Combining Machine Vision and Bayesian Generalization on Concept Hierarchies
Learning a visual concept from a small number of positive examples is a significant challenge for machine learning algorithms. Current methods typically fail to find the appropriate level of generalization in a concept hierarchy for a given set of visual examples. Recent work in cognitive science on Bayesian models of generalization addresses this challenge, but prior results assumed that objects were perfectly recognized. We present an algorithm for learning visual concepts directly from images, using probabilistic predictions generated by visual classifiers as the input to a Bayesian generalization model. As no existing challenge data tests this paradigm, we collect and make available a new, large-scale dataset for visual concept learning using the ImageNet hierarchy as the source of possible concepts, with human annotators to provide ground truth labels as to whether a new image is an instance of each concept using a paradigm similar to that used in experiments studying word learning in children. We compare the performance of our system to several baseline algorithms, and show a significant advantage results from combining visual classifiers with the ability to identify an appropriate level of abstraction using Bayesian generalization.

This paper studies an interesting problem of visual
      concept learning, particularly paying attention to determine the degree to
      which a concept should be generalized. For this purpose, the work first
      focuses on how to generate a large-scale dataset of concepts at various
      levels of abstraction from ILSVRC and collect human judgements on AMT, and
      then presents Bayesian concept learning algorithm to handle perceptual
      uncertainty. The problem of learning concepts of various levels of
      abstraction is interesting. However, this reviewer finds that the
      algorithm presentation is unclear and lacks intuitive explanations. In
      addition, the experimental evaluation is somewhat weak because it doesn’t
      compare with strong baselines. More detailed comments are listed
      as follows: 1. In Eq. (4) and (5), is I(.) an indicator function?
      Please clarity. 2. In Eq. (4), I(x_i \subseteq h) implies that $h$
      is a set and $x_i$ is also a set. This is confusing to this reviewer. Why
      is $x_i$ a set? In contrast, in Eq. (5), I(\hat{y}_i \in h) shows that
      $\hat{y}_i$ is an item. Please clarify. 3. In Eq. (5), it seems
      that the summation is only effective for the first term, i.e.
      A_{j\hat{y}_i}. The remaining part is irrelevant to index $j$. Therefore
      the summation of A_{j\hat{y}_i} becomes one column vector. Is this what
      you want to get? I cannot understand the motivation of this equation.
      4. It seems there is a typo in the last line of page 5. “… the
      true leaf node is $j$ given the classifier output being $j$”. Please check
      if you do want to put two $j$. 5. In line 243, page 5, the
      sentence “for example, as the number of examples that are all Dalmatians
      increases, it becomes increasingly likely that the concept is just
      Dalmatians and not dogs in general even though both are logically
      possible, …” is true. But what does this mean to the hypothesis size? The
      hypothesis is quite confusing in the paper. It seems that the authors
      define the hypothesis size as the number of samples belonging to a
      hypothesis. How can this be defined in real world, given that the number
      of potential images belonging to any concept can be infinitely large? Is
      the number of samples in ImageNet a reasonable way to define the
      hypothesis size? 6. Given so many unclear places in the algorithm
      presentation, it is quite hard to understand the proposed algorithm and
      know why it can really work, although the algorithm presentation is just
      of high level. 7. The baseline methods are not sufficiently strong
      and convincing. For example: a) For the naïve vision approach, only
      GIST feature is used, whereas the proposed algorithm uses a overcomplete
      feature set of 160K dimensions. This is unfair to compare the two
      algorithms. b) The paper mentions the latest work of deep neural
      network [10, 9] in Section 4.3. Why not choose DNN or its variant as a
      baseline method? Simply because it requires more training data? If DNN
      could output the proposed algorithm given that the training data can be
      easily collected from ImageNet, I would doubt the value of this work.


Global Solver and Its Efficient Approximation for Variational Bayesian Low-rank Subspace Clustering
When a probabilistic model and its prior are given, Bayesian learning offers inference with automatic parameter tuning. However, Bayesian learning is often obstructed by computational difficulty: the rigorous Bayesian learning is intractable in many models, and its variational Bayesian (VB) approximation is prone to suffer from local minima. In this paper, we overcome this difficulty for low-rank subspace clustering (LRSC) by providing an exact global solver and its efficient approximation. LRSC extracts a low-dimensional structure of data by embedding samples into the union of low-dimensional subspaces, and its variational Bayesian variant has shown good performance. We first prove a key property that the VB-LRSC model is highly redundant. Thanks to this property, the optimization problem of VB-LRSC can be separated into small subproblems, each of which has only a small number of unknown variables. Our exact global solver relies on another key property that the stationary condition of each subproblem consists of a set of polynomial equations, which is solvable with the homotopy method. For further computational efficiency, we also propose an efficient approximate variant, of which the stationary condition can be written as a polynomial equation with a single variable. Experimental results show the usefulness of our approach.

The problem of low rank subspace clustering can be
      addressed by variational Bayes, with the advantage over MAP methods that
      the rank is adjusted automatically, and virtually no free parameters have
      to be tuned. The posterior variances also possibly give more information.
      As reviewed by the authors (I am not familiar with this
      literature), subspace clustering seems to optimize for an affinity matrix
      to be put into spectral clustering, by essentially regressing datapoints
      onto themselves with low rank constraint. The method here tackles the
      first step, how to obtain the affinity matrix. This can be done by
      variational Bayes. Given that the error is measured by Frobenius norm and
      that all observations are complete (I wonder how realistic that is),
      recent results [18] can be adapted to solve this VB problem by a singular
      value decomp. of the data matrix plus mapping the SVs one by one, using
      some complicated polynomial equations, which however can be solved
      analytically. In particular after a 2nd constraint, the latter is just
      O(L), no iterations are needed. The global optimum is found, even
      including hyperparameters (prior variances). Given that (a)
      subspace clustering is a useful problem, and (b) VB is a good method for
      doing it, the results here are very good and very useful. I cannot
      comment on (a), (b), they cite some papers in computer vision. I am a
      big fan of [18]. What is done here is closely related, but some additional
      new ideas (homotopy method) are needed. Just as with [18], it is easy to
      understand the basic idea just by looking at the Frobenius error, but very
      hard to check the details: I assume they are correct. The
      experimental evaluation is convincing, but they could make a larger effort
      describing the context. What is clustered here? What is the clustering
      error? In order to save space, throw out the equations on iterative VB:
      nobody needs them anymore after this paper! Also, please give proof
      sketches. Here, you do not even state clearly why X diagonalizes in the
      right SV basis -- this is easy to argue for. Also, why did you not
      compare against the simpler MAP solution of (2)? Previous work did not
      seem to employ VB at all, except for [2] (non-empty overlap with the
      authors here?), so I find it odd to only compare against a rather esoteric
      variant -- if another globally solvable method is available.
      Finally, the main limitation of all this work is that I cannot
      have a weighted error, in particular the data matrix Y has to be
      completely observed. It may be good to comment on whether this is an issue
      in the application here.


Thompson Sampling for 1-Dimensional Exponential Family Bandits
Thompson Sampling has been demonstrated in many complex bandit models, however the theoretical guarantees available for the parametric multi-armed bandit are still limited to the Bernoulli case. Here we extend them by proving asymptotic optimality of the algorithm using the Jeffreys prior for 1-dimensional exponential family bandits. Our proof builds on previous work, but also makes extensive use of closed forms for Kullback-Leibler divergence and Fisher information (through the Jeffreys prior) available in an exponential family. This allow us to give a finite time exponential concentration inequality for posterior distributions on exponential families that may be of interest in its own right. Moreover our analysis covers some distributions for which no optimistic algorithm has yet been proposed, including heavy-tailed exponential families.
This quantity measures the expected performance of algorithm A compared to the expected performance of an optimal algorithm given knowledge of the reward distributions, i.e. sampling always from the distribution with the highest expectation.
Since the early 2000s the "optimisim in the face of uncertainty" heuristic has been a popular approach to this problem, providing both simplicity of implementation and finite-time upper bounds on the regret (e.g. [4,7]). However in the last two years there has been renewed interest in the Thompson Sampling heuristic (TS). While this heuristic was first put forward to solve bandit problems eighty years ago in [15], it was not until recently that theoretical analyses of its performance were achieved [1,2,11,13]. In this paper we take a major step towards generalising these analyses to the same level of generality already achieved for "optimistic" algorithms.
Thompson Sampling Unlike optimistic algorithms which are often based on confidence intervals, the Thompson Sampling algorithm, denoted by A π0 uses Bayesian tools and puts a prior distribution π a,0 = π 0 on each parameter θ a . A posterior distribution, π a,t , is then maintained according to the rewards observed in H t−1 . At each time a sample θ a,t is drawn from each posterior π a,t and then the algorithm chooses to sample a t = arg max a∈{1,...,K} {µ(θ a,t )}. Note that actions are sampled according to their posterior probabilities of being optimal.
Our contributions TS has proved to have impressive empirical performances, very close to those of state of the art algorithms such as DMED and KL-UCB [11,9,7]. Furthermore recent works [11,2] have shown that in the special case where each p a is a Bernoulli distribution B(θ a ), TS using a uniform prior over the arms is asymptotically optimal in the sense that it achieves the asymptotic lower bound on the regret provided by Lai and Robbins in [12] (that holds for univariate parametric bandits). As explained in [1,2], Thompson Sampling with uniform prior for Bernoulli rewards can be slightly adapted to deal with bounded rewards. However, there is no notion of asymptotic optimality for this non-parametric family of rewards. In this paper, we extend the optimality property that holds for Bernoulli distributions to more general families of parametric rewards, namely 1-dimensional exponential families if the algorithm uses the Jeffreys prior: Theorem 1. Suppose that the reward distributions belong to a 1-dimensional canonical exponential family and let π J denote the associated Jeffreys prior. Then,
where K(θ, θ ) := KL(p θ , p θ ) is the Kullback-Leibler divergence between p θ and p θ .
This theorem follows directly from Theorem 2. In the proof of this result we provide in Theorem 4 a finite-time, exponential concentration bound for posterior distributions of exponential family random variables, something that to the best of our knowledge is new to the literature and of interest in its own right. Our proof also exploits the connection between the Jeffreys prior, Fisher information and the Kullback-Leibler divergence in exponential families.
Related Work Another line of recent work has focused on distribution-independent bounds for Thompson Sampling. [2] establishes that R(A π U , T ) = O( KT ln(T )) for Thompson Sampling for bounded rewards (with the classic uniform prior π U on the underlying Bernoulli parameter). [14] go beyond the Bernoulli model, and give an upper bound on the Bayes risk (i.e. the regret averaged over the prior) independent of the prior distribution. For the parametric multi-armed bandit with K arms described above, their result states that the regret of Thompson Sampling using a prior π 0 is not too big when averaged over this same prior:
Building on the same ideas, [6] have improved this upper bound to 14 √ KT . In our paper, we rather see the prior used by Thompson Sampling as a tool, and we want therefore to derive regret bounds for any given problem parametrized by θ that depend on this parameter.
[14] also use Thompson Sampling in more general models, like the linear bandit model. Their result is a bound on the Bayes risk that does not depend on the prior, whereas [3] gives a first bound on the regret in this model. Linear bandits consider a possibly infinite number of arms whose mean rewards are linearly related by a single, unknown coefficient vector. Once again, the analysis in [3] encounters the problem of describing the concentration of posterior distributions. However by using a conjugate normal prior, they can employ explicit concentration bounds available for Normal distributions to complete their argument.
Paper Structure In Section 2 we describe important features of the one-dimensional canonical exponential families we consider, including closed-form expression for KL-divergences and the Jeffreys' prior. Section 3 gives statements of the main results, and provides the proof of the regret bound. Section 4 proves the posterior concentration result used in the proof of the regret bound.

The paper provides a logarithmic regret bound for a
      known bandit algorithm, Thompson Sampling, in the case of bandit problems
      with reward distributions being part of the exponential family. The result
      is novel and generalizes previous work done for the case of Bernoulli
      bandits. Quality: The paper seems technically sound. The
      technical results are interesting and the proofs are non-trivial.
      One key difference with previous work is the change of the prior
      distribution for Thompson Sampling (from a uniform prior to Jeffreys
      prior). The authors provide reasons as to why the Jeffreys prior should be
      used since it helps the theoretical analysis. My first question is though,
      what would be the motivation in an applied setting to prefer a Jeffreys to
      a uniform prior? And what would be the practical impact of this change?
      For example, in the case of a Bernoulli distribution, the Jeffreys prior
      is Beta(0.5,0.5) which encodes a prior belief that the optimal expected
      reward is either closer to 0 or closer to 1. This seems unreasonable if no
      prior knowledge exists about the arms. My second question is about
      the proof of inequality (6) in Appendix A. At line 517, D_{a,s} is defined
      to be the complement of the second part of the event E_{a,t} defined in
      line 219 (equation (5)). But D_{a,s} is not really the complement (as it
      would require an \forall s' quantifier instead of the existential
      quantifier). I am assuming the authors skipped some steps, so could you
      please clarify? Clarity: The paper is relatively difficult to
      read. One problem is the disproportionate use of mathematical notation at
      the expense of providing intuitions for the used variables and the
      results. Here are several examples: - L(\theta) (introduced in
      line 216), while an important variable for the proof, has no explanation
      as to why it is defined that way other than it is a "likely realization of
      arm a". - The superscript 'c' used for the first time in line 244 and
      used through the proof has no explanation - the reader is forced to infer
      that the authors mean the complement of the event E_{a,t}. - The
      quantity t^b introduced in line 247 is not defined and it is only
      introduced later in the text. - A lot of subscripts and superscripts
      (like TS, pi_J) could easily be removed with the benefit of improving the
      readability of the results. Regarding the organization, I like
      that the authors provide the high level argument "graph" before the
      proofs, but I think the paper could be improved if this high level
      description would be more intuitive even at the cost of moving more
      technical details to the appendix. Another concrete example of
      what could help the readability is a corollary of Theorem 4 that would
      make it explicit for a particular distribution class. Originality
      and Significance: I agree with the authors that Theorem 4, that
      introduces a new tool for proving concentration bounds for posterior
      distributions, is new (to the best of my knowledge) and interesting in
      itself. The high level strategy of the proof for Theorem 2 (the
      main result of the paper) follows the idea of the proof for the Bernoulli
      bandit case in Kaufman et al 2012 while being more technically involved
      and general.


Scalable kernels for graphs with continuous attributes
While graphs with continuous node attributes arise in many applications, stateof-the-art graph kernels for comparing continuous-attributed graphs suffer from a high runtime complexity. For instance, the popular shortest path kernel scales as O(n 4 ), where n is the number of nodes. In this paper, we present a class of graph kernels with computational complexity O(n 2 (m + log n + δ 2 + d)), where δ is the graph diameter, m is the number of edges, and d is the dimension of the node attributes. Due to the sparsity and small diameter of real-world graphs, these kernels typically scale comfortably to large graphs. In our experiments, the presented kernels outperform state-of-the-art kernels in terms of speed and accuracy on classification benchmark datasets.

This paper proposed a new kernel function and its fast
      computation algorithm for graphs with continuous attribute values. The
      proposed algorithm is much more efficient than competitive methods. The
      advantages of proposed method was proved by theoretically and evaluated by
      numerical experiments. Quality: The efficiency of the proposed
      method was through investigated both by theory (proved the computational
      complexity) and by numerical experiments, which shows the proposed method
      is much better than related works. But the new kernel function itself was
      not analyzed enough, which was only evaluated by performance results on
      classification tasks. One might have the question which pairs of graphs
      are similar/dissimilar on the proposed kernel (similarity measure). The
      comparison with other kernel functions using toy graph examples would be
      necessary. Clarity: The paper is well organized and
      understandable for readers. Originality: This paper proposed a
      novel method, with a new kernel for graphs and furthermore proposed a fast
      computation algorithm for the new kernel. The proposed method can manage
      massive datasets of graphs with continuous attribute values, which cannot
      be managed by conventional methods. Significance: The proposed
      method and results would be interesting. Because the result is general,
      this method could be applicable in a lot of applications.
      Other comments: 1. The proposed computation might
      evaluate the kernel values for same pairs of paths multiple times, which
      means that the algorithm might assume the path from $v_a$ to $v_b$ and the
      path from $v_b$ to $v_a$ as different paths. If it is true, analyzing how
      it could affect to the similarity measure would be important. 2.
      The caption in Table 2 should mention the meaning of the real values
      even if it is mentioned in the main sentence. And, what are the values in
      the parentheses in the row of SP? If it is average runtimes, they are less
      than 24 hours. Then the runtime of SP should not be OUT OF TIME.


Sparse Precision Matrix Estimation with Calibration
We propose a semiparametric method for estimating sparse precision matrix of high dimensional elliptical distribution. The proposed method calibrates regularizations when estimating each column of the precision matrix. Thus it not only is asymptotically tuning free, but also achieves an improved finite sample performance. Theoretically, we prove that the proposed method achieves the parametric rates of convergence in both parameter estimation and model selection. We present numerical results on both simulated and real datasets to support our theory and illustrate the effectiveness of the proposed estimator.

Summary of the paper This paper presents a
      semi-parametric tuning-free procedure for estimating sparse concentration
      matrices. This method is applicable to the elliptical distribution family,
      while most of its competitors only apply to the sub-Gaussian distribution
      family. The procedure, called ALICE, learns the precision matrix column by
      column in a similar fashion than the CLIME (Cai et al, 2011), yet with
      important modifications: a first step is designed to learn the correlation
      matrix and the associated variances/standard deviations by means of the
      Kendall's Tau statistic as proposed in Liu et al, 2012. Then, the standard
      deviations are estimated through a recent proposal of Catoni (2012). In
      the second step, the inverse correlation is recovered by plugin-in the
      correlation estimated in the first step in a convex program similar to the
      CLIME, yet with a modification that allows for some calibration between
      the columns. This leads to the tuning-free property of the ALICE. Finally,
      the third step recovers the inverse covariance matrix after a simple
      rescaling of the inverse correlation matrix. A dual inexact iterative
      projection algorithm is given to (approximately) solve the convex program
      in step 2. Theoretical guarantees, in both terms of estimation consistency
      and selection consistency are provided, equivalent to those proposed for
      the CLIME estimator. A simulation study on synthetic and breast cancer
      data illustrates the good performance of the method compared to the
      state-of-the-art methods. Comments The introductory part
      in this work gives a concise and appropriate bibliography. Motivations for
      semi-parametric estimators and limitations of the state-of-the art methods
      are clearly introduced. Contributions of the current proposal (ALICE) are
      clearly stated. When the method (part 3) is presented though, some
      additional justification and explanation could have helped the reading and
      the general understanding: why choosing Kendall's Tau + Catoni's M
      estimator? Any other possibilities? It is also not very clear what is due
      to the authors: using Kendall's Tau for correlation estimation in the
      elliptical distribution framework is apparently due to a previous work
      (Liu et al, 2012b): what of Catoni's M estimator? Is this already used in
      such a context? The convex program solved looks like a
      modification of the CLIME criterion. Some more connexion at this stage of
      the paper (part 3.2) with the CLIME would have been appropriate. As a
      matter of fact, it would have been easier to judge the originality of the
      algorithm that follows, whose presentation is somewhat cumbersome. The
      algorithm gives an approximation of the target estimator, but no
      quantification are made on this approximation. Providing a global
      complexity of the procedure would have been a good idea, too. The
      same kind of remarks go for the theoretical part: though quite nice, the
      properties derived by the authors are close to the properties existing for
      the CLIME; and it is hard to evaluate how straightforward it can be
      deduced from the work of Cai et al. Regarding the numerical
      experiments, the good behavior of ALICE on non strictly Gaussian data is
      nicely illustrated. Still, I do not understand why the proposal of Liu et
      al., 2012b, is not included in the analysis. The same for the GLASSO with
      a prior transformation of the sample covariance matrix, as implemented in
      the huge package with the non paranormal transformation. It is a
      pity that no illustration is made of the tuning-free property of ALICE,
      since the calibration parameters are tuned with cross-validation in the
      numerical experiments.


Reciprocally Coupled Local Estimators Implement Bayesian Information Integration Distributively
Psychophysical experiments have demonstrated that the brain integrates information from multiple sensory cues in a near Bayesian optimal manner. The present study proposes a novel mechanism to achieve this. We consider two reciprocally connected networks, mimicking the integration of heading direction information between the dorsal medial superior temporal (MSTd) and the ventral intraparietal (VIP) areas. Each network serves as a local estimator and receives an independent cue, either the visual or the vestibular, as direct input for the external stimulus. We find that positive reciprocal interactions can improve the decoding accuracy of each individual network as if it implements Bayesian inference from two cues. Our model successfully explains the experimental finding that both MSTd and VIP achieve Bayesian multisensory integration, though each of them only receives a single cue as direct external input. Our result suggests that the brain may implement optimal information integration distributively at each local estimator through the reciprocal connections between cortical regions.

"Reciprocally Coupled Local Estimators Implement
      Bayesian Information Integration Distributively" puts forward a new take
      on Bayesian integration of multimodal cues. Instead of assuming a special
      area in the brain, where evidence from various sensory cues is combined
      (as in Ma and all, 2006), the authors consider a scenario, whereby each
      area receiving direct afferent input from a single modality (i.e. visual)
      combines information from other modalities (e.g. vestibular) via
      reciprocal connections. In the example analysed by the authors, and
      under a number of suitable assumptions, the cue integration they observe
      in their networks is close to Bayes-optimal. Building up on work
      of Fung and all (2010), the authors derive theoretical predictions for the
      integration of information in reciprocally coupled ring attractors
      (CANNs), which they also confirm by simulations. The reader is led through
      the general steps of the analysis, while details are provided in the
      supplementary material. The main result is that for a sufficiently
      weak input, and sufficiently narrow interactions (HWHH = 67 degrees for
      their example of heading direction), each of the networks combines
      information from the two cues in a Bayes optimal manner. That is, the
      decoding precision of a single network when both cues are available is a
      sum of this network's precisions when only a single cue is present, i.e.
      p(theta| c1, c2) = p1(theta|c1) + p1(theta|c2), where c1 is either
      - the directly encoded cue, e.g. visual, which has a higher decoding
      accuracy, or - the other modality cue, reported via reciprocal
      connections, with decoding accuracy growing with the strength of
      reciprocal connections. This result is interesting despite the
      simplifying assumptions. My only major concern would be the lack of
      attention given to the posterior mean. As indicated by Fig. 3C, if the
      cues were not congruent, the networks would display bias towards the mean
      of the two cues larger than predicted by Bayesian inference. Although
      presented, this effect is not discussed anywhere in the article. Likewise,
      the Bayes estimate of the mean could be plotted together with the results
      of the simulations in Fig. 4. Quality This article is
      technically sound, supported well by theoretical predictions and
      simulations. It would be possible to evaluate weaknesses of the work in
      more detail (see above). Clarity The article is clearly
      written, although it would benefit from one more careful reading. In
      Fig. 4 legend, values of mu_1 and mu_2 seem to be swapped. L. 395,
      reference to Eq. 3 was possibly meant to be Eq. 2. Originality
      To the best of my knowledge, the approach is new. The work it builds
      upon is well referenced. Significance As above, I believe the
      impact of this work will reach a wide audience, far wider than modellers
      working with attractor networks.


Flexible sampling of discrete data correlations without the marginal distributions
Learning the joint dependence of discrete variables is a fundamental problem in machine learning, with many applications including prediction, clustering and dimensionality reduction. More recently, the framework of copula modeling has gained popularity due to its modular parameterization of joint distributions. Among other properties, copulas provide a recipe for combining flexible models for univariate marginal distributions with parametric families suitable for potentially high dimensional dependence structures. More radically, the extended rank likelihood approach of Hoff (2007) bypasses learning marginal models completely when such information is ancillary to the learning task at hand as in, e.g., standard dimensionality reduction problems or copula parameter estimation. The main idea is to represent data by their observable rank statistics, ignoring any other information from the marginals. Inference is typically done in a Bayesian framework with Gaussian copulas, and it is complicated by the fact this implies sampling within a space where the number of constraints increases quadratically with the number of data points. The result is slow mixing when using off-the-shelf Gibbs sampling. We present an efficient algorithm based on recent advances on constrained Hamiltonian Markov chain Monte Carlo that is simple to implement and does not require paying for a quadratic cost in sample size.

This paper is concerned with learning Gaussian copulas
      after the empirical cdf is used to transform the marginals of all data.
      The challenge in this scenario seems to be linear constraints on the
      sampling domain, the number of which increases quadratically with the
      number of data points. The paper proposes a Hamiltonian Monte Carlo
      sampler where the constraints are handled by the sample trajectory
      "bouncing" off them. Empirical results are given, in particular showing
      superior performance, for the same number of steps, against the Hoff
      algorithm. The paper seems technically sound, although I have some
      concerns regarding the empirical results. The presentation is fine,
      although note minor points below. I have not seen an approach such as this
      before and believe it to be original, although am not well-abreast of the
      copula literature, and my confidence score reflects this. The approach
      does seem quite computationally expensive, and the authors even admit that
      their MATLAB implementation is not particularly fast, which leaves
      question marks over potential impact. My concerns with the
      empirical results are: * The HMC is only run for 1000 steps, while
      the Hoff is run for 2000. The Hoff is clearly seen to make a few switches
      after the 1000 step mark, and a reader might worry that the HMC method
      could do the same. I presume this has been done because the HMC is much
      more expensive to run (the authors say 50 times so in Footnote 7). I don't
      doubt the final result, especially looking at the plots in the
      supplementary material, but for thoroughness it would be good to see the
      HMC run for 2000 steps, or a plot of posterior densities of each sample
      that indicates that the jumps of the Hoff algorithm are not due to a
      second mode, or a discussion point affirming that the posterior is
      definitely unimodal (which I think is knowable in this case a priori).
      * I'd like to see more discussion on the relative computational
      expense of the HMC and Hoff methods. Footnote 7 makes an excuse, whereas
      I'd rather see an explanation. Should I expect a good implementation of
      HMC to be as fast as Hoff? At the moment, with HMC taking 50 times longer,
      it may be fairer to compare HMC at iteration #50 with Hoff at iteration
      #2500 in the bottom row of Figure 3. A few minor points on
      presentation: * There's a citation, [12], given for the first time
      in the conclusion. If this is important it should probably be introduced
      as related work in the introduction. * The footnoting might be
      considered excessive. I'd suggest that some of these could be moved into
      the text without distracting the reader, especially 2, as it ends a
      paragraph anyway, but 3, 5, and 7 as well. In fact 5 should be, as it
      introduces notation, which is required knowledge rather than an aside, and
      I think 7 also, as, in the context of comments on the empirical results
      above, it's important to know that the HMC steps take a lot longer to run.


Manifold-based Similarity Adaptation for Label Propagation
Label propagation is one of the state-of-the-art methods for semi-supervised learning, which estimates labels by propagating label information through a graph. Label propagation assumes that data points (nodes) connected in a graph should have similar labels. Consequently, the label estimation heavily depends on edge weights in a graph which represent similarity of each node pair. We propose a method for a graph to capture the manifold structure of input features using edge weights parameterized by a similarity function. In this approach, edge weights represent both similarity and local reconstruction weight simultaneously, both being reasonable for label propagation. For further justification, we provide analytical considerations including an interpretation as a cross-validation of a propagation model in the feature space, and an error analysis based on a low dimensional manifold model. Experimental results demonstrated the effectiveness of our approach both in synthetic and real datasets.

The manuscript a new method “adaptive edge weighting”
      (AEW) that weights to the edges of a neighborhood graph for use with label
      propagation. The neighborhood graph is constructed using k-NN in the
      feature space, and then the edge weights are optimized based on their
      ability to reconstruct the feature vector associated with each node. The
      objective function is the same as LLE but the weights are derived from
      Gaussian kernels and the only parameters are the bandwidths of the kernels
      in each dimension, thereby forcing the weights to be positive and greatly
      reducing the number of free parameters. They call this procedure “feature
      propagation”. The manuscript contains a theorem that connects the
      reconstruction error in feature propagation (and the maximum neighbor
      distance) with the expected performance in the label propagation task.
      When tested on simulated data or eight classification / multiclass
      benchmarks and paired with one flavor of label propagation (Harmonic
      Gaussian Fields), AEW performs better than a version that does not
      optimize bandwidths or a previous, similar method (LNP) that uses the
      unrestricted LLE objective function to set the edge weights. The
      manuscript also points out that the method is less sensitive to increases
      in k when constructing the neighborhood graph because of the Gaussian
      kernel derived edge weights. Clarity: The manuscript is
      well-organized and easy to follow. The experimental section is clear
      although one very minor issue is that I am not sure that I could perfectly
      reproduce their results, as the exact optimization procedure they use for
      the bandwidths is not described (steepest descent? CG?) and the synthetic
      data set is not provided. Quality: This is a well-constructed,
      good quality paper – it provides a new methodology, some insight into why
      this method is appropriate through the aforementioned theorem, as well as
      sufficient experimental evaluation. I have some questions about the
      validation. LNP appears to use a different neighborhood graph than AEW,
      how much of the performance improvement of AEW is due to the different
      graphs? Also, AEW doesn’t generate a normalized Laplacian (does LNP?), but
      normalized version of the AEW Laplacian was used in the validations,
      presumably to improve performance. This issue is only discussed briefly; I
      think some more discussion is warranted. Originality: The
      contribution seems original although is a slight, but crafty, modification
      of a previous technique (LNP) for setting the edge weights.
      Significance: The LNP paper has >200 Google scholar citations
      and the described method performs better, is relatively easy to implement,
      and has no tuning parameters. Given a relevant learning problem, this
      would be the first method that I would try. So, it does have some
      significance. On the other hand, this does seem like an incremental
      improvement on well-studied problem although the theorem does provide a
      basis on which to develop other “feature propagation” based weighting
      schemes. One other issue that could be addressed in the introduction to
      increase the significance of the work, is why one would want to convert
      feature vectors into a graph rather than working directly in the feature
      space. Minor issues: Line 119: Are the limits of the argmax
      correct here? “argmaxj≤j Fij “


Bayesian Mixture Modeling and Inference based Thompson Sampling in Monte-Carlo Tree Search
Monte-Carlo tree search (MCTS) has been drawing great interest in recent years for planning and learning under uncertainty. One of the key challenges is the trade-off between exploration and exploitation. To address this, we present a novel approach for MCTS using Bayesian mixture modeling and inference based Thompson sampling and apply it to the problem of online planning in MDPs. Our algorithm, named Dirichlet-NormalGamma MCTS (DNG-MCTS), models the uncertainty of the accumulated reward for actions in the search tree as a mixture of Normal distributions. We perform inferences on the mixture in Bayesian settings by choosing conjugate priors in the form of combinations of Dirichlet and NormalGamma distributions and select the best action at each decision node using Thompson sampling. Experimental results confirm that our algorithm advances the state-of-the-art UCT approach with better values on several benchmark problems.

The paper proposes a Bayesian inference in Monte-Carlo
      tree search (MCTS) with Thompson sampling based action-selection strategy,
      called Dirichlet-NormalGamma MCTS (DNG-MTCS) algorithm. The method
      approximates the accumulated reward of following the current policy from a
      state, X_{s,\pi(s)}, by the normal distribution with the NromalGamma
      distribution prior. The state transition probabilities are estimated via
      Dirichlet distributions. Action-selection strategy is based on Thompson
      sampling approach, where the expected cumulative reward for each action is
      computed with the parametric distribution with parameters drawn from the
      posterior distributions and then the action with the highest expectation
      is selected. The authors apply the proposed method to several benchmark
      tasks and showed that the method can converge (slightly) faster than the
      UCT algorithm. Theoretical properties about convergence are also provided.
      Although there is a lot of work on MCTS, I am not aware of other
      work based on the specific approach made in the paper. However, the
      density approximation of the cumulative reward with NormalGamma prior is
      not new and has been proposed by Dearden et al. 1998 [a]. Modeling of the
      state transition with Dirichlet prior is well known. So the originality of
      the proposed modeling and inference methods in Section 3.2 will be small.
      However, the proposed combination with this modeling and action-selection
      strategy based on Thompson sampling is new, to the best of my knowledge,
      and might be of interest to related areas of research. The
      presented experimental result would be week. Although there is a lot of
      work on MCTS, the applied baseline method was limited to the classic UCT
      (except for the CTP problem). More detail experiments in terms of
      computational costs as well as sample complexity with several advanced
      methods in [b,d,e] will be needed. Most part of this paper is
      clearly written. But the assumptions in Section 3.1 are not quite clear. I
      feel that there is a contradiction. Although the cumulative reward given
      (s, a), X_{s,a}, is modeled as a mixture of Normal distributions as a
      result, the authors argue the assumption of X_{s,\pi(s)} having a Normal
      distribution, is realistic. However, in what some see as, X_{s,\pi(s)} has
      to be a mixture of Normal distributions and the number of mixtures will
      increase exponentially (from leaf states to root). So their assumption of
      X_{s,\pi(s)} cannot be realistic. But I think it should be one of
      practical approaches for approximating the distribution of X_{s,\pi(s)}.
      The relation between cumulative rewards in successive time-steps would be
      related to the distributional Bellman equation in [c]. I’d like to
      recommend revising this part carefully in the final version. In
      line 137--139, N(n\mu, \delta^2/n) is N(n\mu, n\delta^2)? Doesn't the
      \sum_n f diverge as n \to \infty? [a] Richard Dearden, Nir
      Friedman, Stuart J. Russell: Bayesian Q-Learning. AAAI/IAAI 1998. [b]
      Gerald Tesauro, V. T. Rajan, Richard Segal: Bayesian Inference in
      Monte-Carlo Tree Search. UAI 2010. [c] Tetsuro Morimura, Masashi
      Sugiyama, Hisashi Kashima, Hirotaka Hachiya, Toshiyuki Tanaka: Parametric
      Return Density Estimation for Reinforcement Learning. UAI 2010. [d]
      John Asmuth and Michael L. Littman. Approaching Bayes-optimalilty using
      Monte-Carlo tree search. In Proc. 21st Int. Conf. Automat. Plan. Sched.,
      2011. [e] Arthur Guez, David Silver, Peter Dayan: Efficient
      Bayes-Adaptive Reinforcement Learning using Sample-Based Search. NIPS
      2012.


Discriminative Transfer Learning with Tree-based Priors
High capacity classifiers, such as deep neural networks, often struggle on classes that have very few training examples. We propose a method for improving classification performance for such classes by discovering similar classes and transferring knowledge among them. Our method learns to organize the classes into a tree hierarchy. This tree structure imposes a prior over the classifier's parameters. We show that the performance of deep neural networks can be improved by applying these priors to the weights in the last layer. Our method combines the strength of discriminatively trained deep neural networks, which typically require large amounts of training data, with tree-based priors, making deep neural networks work well on infrequent classes as well. We also propose an algorithm for learning the underlying tree structure. Starting from an initial pre-specified tree, this algorithm modifies the tree to make it more pertinent to the task being solved, for example, removing semantic relationships in favour of visual ones for an image classification task. Our method achieves state-of-the-art classification results on the CIFAR-100 image data set and the MIR Flickr image-text data set.

The paper presents a multi-task learning approach
      based on hierarchical Bayesian priors. The priors are combined with a deep
      NN resulting in a discriminative learning process which encourages sharing
      information among tasks taking into account different levels of
      relatedness. The paper is written well and easy to follow. The
      topic of multi-task learning is a very popular and important topic and the
      sub issue of considering different level of relatedness has also gained
      much attention lately. I liked the proposed integration with NN and the
      good experimental results. I feel the paper does not give enough
      background on related approaches. There are many related papers, but
      specifically I feel a discussion on similar hierarchical Bayesian
      approaches is lacking: Bayesian Multitask Learning with Latent
      Hierarchies, H. Daume III where a similar prior is seemed to be
      used for what they denote the domain adaptation approach. Further
      related work is: Multi-task learning for classification with
      Dirichlet process priors, Xue et al. 2007 and more papers
      presenting discriminative approaches dealing with different levels of
      relatedness: Learning with whom to share in multi-task feature
      learning, kang et al. Hierarchical regularization cascade for
      joint learning, zweig et al. Tree-guided group lasso for
      multi-task regression with structured sparsity. Kim et al. I
      also have a few questions regarding the algorithm for discovering the
      hierarchical structure: - The hierarchy discovery is described in
      the context of the CRP prior but then it is said that solving the
      inference problem directly is too complex and thus an approximation is
      proposed. How does this approximation relate to the original prior? I
      couldn't find the explicit use of the prior (over the generation of a new
      super class) in the presented approach. - Unless I missed
      something, it seems that the current presentation and experiments deals
      only with a two level hierarchy, super class and original classes. Can the
      approximation method be at all extended to more levels? - The
      method requires a validation set in order to decide on the best model at
      each step of the hierarchy construction algorithm. Can we assume a
      validation set in small sample scenarios (which are really the motivating
      scenario for multi-task learning)? I saw in the supplementary material
      this issue is addressed, but didnt understand if the validation set
      discussed there is the validation set for the hyper-parameters or the
      hierarchy discovery or both. - How is the convergence of the
      hierarchical structure measured? - The algorithm assumes an
      initial good guess. What can be done when such a guess is not available or
      too costly? would a random initialization work well? It would be nice to
      see an experiment with a random initialization Generally I
      liked the experimental section and found it sufficient. My only concern is
      that I would have preferred to see the experiment on small sample of only
      a single class done on more examples, then only the dolphin class.


Mid-level Visual Element Discovery as Discriminative Mode Seeking
Recent work on mid-level visual representations aims to capture information at the level of complexity higher than typical "visual words", but lower than full-blown semantic objects. Several approaches [5,6,12,23] have been proposed to discover mid-level visual elements, that are both 1) representative, i.e., frequently occurring within a visual dataset, and 2) visually discriminative. However, the current approaches are rather ad hoc and difficult to analyze and evaluate. In this work, we pose visual element discovery as discriminative mode seeking, drawing connections to the the well-known and well-studied mean-shift algorithm [2,1,4,8]. Given a weakly-labeled image collection, our method discovers visually-coherent patch clusters that are maximally discriminative with respect to the labels. One advantage of our formulation is that it requires only a single pass through the data. We also propose the Purity-Coverage plot as a principled way of experimentally analyzing and evaluating different visual discovery approaches, and compare our method against prior work on the Paris Street View dataset of [5]. We also evaluate our method on the task of scene classification, demonstrating state-of-the-art performance on the MIT Scene-67 dataset.

The paper proposes a new method for learning
      discriminative image patches predictive of the image class. The procedure
      starts by considering all (?) patches in an image collection. The
      discriminative patches are then found as the centres of patch clusters,
      obtained by a discriminative version of the mean-shift algorithm.
      Discriminativeness is incorporated in mean shift by dividing the kernel
      density estimate of the positive patches by the one of negative ones.
      Pros: - The problem of learning discriminative
      patches/parts is an important one. - The classification performance of
      the proposed discriminative patches is very good, at least on MIT Scene
      67. - There are a few interesting insights in the formulation.
      Cons: - The authors start from mean-shift and gradually
      transform it into a very different algorithm. In the end, I am not sure
      that the mean-shift interpretation is useful at all. - Several aspects
      of the formulation and derivation are heuristics. Some of the heuristics
      are changed on a dataset-by-dataset basis. - The learning algorithm is
      also heuristic, with no formal guarantee of correctness/convergence. -
      There are several small errors in the formal derivation. Detailed
      comments: The empirical results are sufficiently strong that this
      paper should be considered for publication. However, the formulation and
      technical derivation should be improved significantly, as detailed below:
      ### l.125: The notation should be clarified. The expression
      max(d(x_i,w) - b, 0) is the triangular kernel assuming that d() is the
      negative of the Euclidean distance (as stated on l. 130) *and* that the
      bandwidth b is negative, which is counter-intuitive. max(b - d(x_i,w), 0)
      is more natural. ### Eq. (1). arglocalmax is never defined.
      ### Eq. (2). This equation indicates that, contrary to what
      stated in the manuscript, the algorithm is not maximizing a ratio of
      density values, but an energy function computed with a sort of adaptive
      bandwidth. This density is given by E(w) = sum_i max(d(x_i^+) -
      b(w), 0) where the bandwidth b(w) is selected as a function of the
      current point w as b(w) = b : sum_i max(d(x_i^+) - b, 0) =
      epsilon. Several aspects of this formulation should be clarified:
      (a) The triangular kernel should be normalized by its mass to get
      a proper density estimator. Interestingly, this normalization factor,
      which depends on w, cancels out in the ratio (2), which perhaps "saves the
      day". (b) This formulation should be contrasted to the standard
      adaptive mean shift (e.g. B. Georgescu, I. Shimshoni, and P. Meer. Mean
      shift based clustering in high dimensions: A texture classification
      example. In Proc. ICCV, 2003). There the bandwidth is chosen as a function
      of x_i rather than w and the normalization of the kernels become crucial.
      (c) What happens if there are more than one value of b(w)
      satisfying the constraint in (2) ? ### Eq. (3) l. 157:
      It is the _squared_ Euclidean distance d^2() that reduces to the inner
      product, not the euclidean distance d(). Note that restricting the
      domain to the unit sphere requires in principle to modify all the
      densities to have this manifold as domain. Fortunately, the required
      modification (normalising factors) does not seem to have a consequence in
      this case. l. 177: it seems to me that changing lambda _does_
      change the solution w, not just its norm. To keep the direction of w
      invariant while changing lambda, epsilon must change as well. Therefore,
      choosing different values of lambda should have an effect on the solution,
      unless all values of epsilon are equally good (but then why having epsilon
      in the first place?). ### Eq. (5) This is where the
      proposed method diverges substantially from mean shift. Mean shift applies
      hill climbing to each w independently starting from w = x_i for all data
      points, in order to determine a clustering of the data x_i themselves.
      Here, instead, the authors formulate (5) and (6) as a method to "explain"
      all the data. Practical differences include: - mean-shift is
      non-parametric, in the sense that the number of clusters is not specified
      a priori. Here the authors start with a fixed number of cluster centers
      w1...wK and optimise those to fit the data, which is more similar to
      K-means. - the authors worry about the fact that "patches should
      not be double counted" and introduce a set of soft associations
      data-cluster alpha_ij. This is difficult to map in the standard semantic
      of mean-shift clustering, where the association of a data point x_i to a
      mode wk is obtained implicitly by the locality of the kernel estimator
      only. As the authors argue, alpha_ij establish a "competition" between
      modes to explain data points, which again is more similar to k-means.
      - the way the alpha_ij are updated has little to do with the
      optimization of (6) and is completely ad hoc (l.199 - 211) ###
      Optimization method Unfortunately this method is just an heuristic
      (l. 212-259). ### Experiments Baselines: [5,8]
      have mechanism to avoid or remove redundant patches, which do not seem to
      be incorporated in this baseline. Removing such redundant patches might
      affect Fig. 3, 4. Scene-67 experiments: There are several tuning
      of the representation (e.g. number of HOG cells in a descriptor) that
      probably helps the method achieve state of the art results. While this is
      ok, the authors should consider re-running this experiment with the
      baseline discriminative patches obtained as in Fig. 4. ###
      Other minor comments l.315: I have seen [5] and [8] in CVPR 2013
      and it seems to me that they both have LDA
      retraining.


Deep Fisher Networks for Large-Scale Image Classification
As massively parallel computations have become broadly available with modern GPUs, deep architectures trained on very large datasets have risen in popularity. Discriminatively trained convolutional neural networks, in particular, were recently shown to yield state-of-the-art performance in challenging image classification benchmarks such as ImageNet. However, elements of these architectures are similar to standard hand-crafted representations used in computer vision. In this paper, we explore the extent of this analogy, proposing a version of the stateof-the-art Fisher vector image encoding that can be stacked in multiple layers. This architecture significantly improves on standard Fisher vectors, and obtains competitive results with deep convolutional networks at a smaller computational learning cost. Our hybrid architecture allows us to assess how the performance of a conventional hand-crafted image classification pipeline changes with increased depth. We also show that convolutional networks and Fisher vector encodings are complementary in the sense that their combination further improves the accuracy.

This paper uses Fisher Vectors as inner building
      blocks in a recognition architecture. The basic Fisher vector module had
      previously demonstrated superior performance in recognition application.
      Here, it is augmented with discriminative linear projection for
      dimensionality reduction, and multiscale local pooling, to make it
      suitable for stacking. Inputs of all layers are jointly used for
      classification. Experiments show convincing performance on ImageNet.
      This paper is well organized, well written, and shows good results,
      establishing that (1) adding another layer of Fisher vector extraction is
      useful, (2) using both layers jointly is better than using any one of them
      in isolation (this confirms earlier work, as stated in the paper), (3)
      using multiscale pooling is better than using a single pooling
      neighborhood size. Concerning this last point: the idea of multiple
      pooling scales vs. a single one is analogous to the difference between
      spatial pyramid and simple bag of words, so this is not really new, and
      should be mentioned. A small gripe with the writing is with two ways
      that the paper is sold: (1) the conclusion reads, "the architecture is
      largely hand-crafted, and requires lighter training." Why is using
      hand-crafted features seen as a good thing? The fact that other deep
      networks incorporate training allows adaptation to other domains. For a
      vision application, training could be avoided by plugging in previously
      trained weights, using training samples as weights, using only
      unsupervised pre-training, etc. -- the option to fully train a network
      simply results in the best performance, but the amount of needed training
      is flexible if one is willing to accept traning-performance trade-offs.
      (2) the introduction and conclusion take aim at a strawman: e.g. in
      the introduction, l.48: "are the deep architectures used in CNN really
      necessary, or can comparable performance be obtained simply by stacking
      the well crafted features developed in the computer vision literature?".
      This might unnecessarily alienate the deep learning crowd, given that it
      has so often been emphasized that all types of modules are suitable for
      embedding in deep learning architectures, and bits and pieces of computer
      vision architectures have in fact been used (e.g. see work by Weston and
      Collobert; the angle taken by Coates et al. in ref. 22; the use of SVM and
      spatial pyramids mixed with convolutional network layers in Kavukcuoglu et
      al. NIPS 2010.) Instead, a better way to describe the contribution of the
      paper could be to reverse the framing and ask whether conventional
      computer vision features could benefit from the stacking principles used
      in deep architectures. Minor comments: Ref. 7 used for instances of
      sparse coding and max pooling should be replaced by earlier work, e.g.:
      Yang et al. CVPR 2009, Linear spatial pyramid matching using
      sparse coding for image classification


Nonparametric Multi-group Membership Model for Dynamic Networks
Relational data-like graphs, networks, and matrices-is often dynamic, where the relational structure evolves over time. A fundamental problem in the analysis of time-varying network data is to extract a summary of the common structure and the dynamics of the underlying relations between the entities. Here we build on the intuition that changes in the network structure are driven by dynamics at the level of groups of nodes. We propose a nonparametric multi-group membership model for dynamic networks. Our model contains three main components: We model the birth and death of individual groups with respect to the dynamics of the network structure via a distance dependent Indian Buffet Process. We capture the evolution of individual node group memberships via a Factorial Hidden Markov model. And, we explain the dynamics of the network structure by explicitly modeling the connectivity structure of groups. We demonstrate our model's capability of identifying the dynamics of latent groups in a number of different types of network data. Experimental results show that our model provides improved predictive performance over existing dynamic network models on future network forecasting and missing link prediction.

Nonparametric Multi-group Membership Model for Dynamic
      Networks The paper proposes a new time varying multiple-membership
      network model. The multiple group membership is modelled using an IBP
      and the time evolution is modelled by a hidden Markov model. In
      addition, groups can be born or die according to a stochastic process.
      Modeling relational data using Bayesian nonparametrics has
      recently received a lot of attention, and this paper makes a small but
      significant step towards better modeling the dynamic behaviour of
      evolving networks. In my mind, the proposed model makes
      perfect sense as a general and very expressive model of time varying
      networks. The only thing that I have some trouble understanding is the
      argument for having the birth death process in the model. At every
      time-step there are an infinite number of groups (according to the
      IBP), and a finite number of the groups have members associated with
      them. The HMMs tie these IBPs together, and if all members at some
      point leave a group it will just be empty. Later, members can again
      join this empty group (or join another empty group). These two
      situations are disambiguated by the group parameters (link affinities)
      so the inference will determine wheter a new group should be formed or
      the old group should be "reused". The exact same is the case for the
      model with the birth-death process, only this is a more complicated
      model. Perhaps I misunderstand something? I think the paper is
      very well written, and the argumentation (except the above mentioned)
      was easy to follow. There is a nice balance between background
      information, model specification, inference details, and experimental
      validation. Some details regarding the inference is left out
      (available in an extended version which I did not examine) and I think
      this is a reasonable choice, since this is not essential for
      understanding the main arguments. A few additional comments:
      - The proposed model, like the LFRM model of Miller et al. [23] is
      based on a logistic link function which is the reason that the
      computational complexity of inference is at least N^2. I suspect
      this prohibits the use of the proposed model to medium or large
      scale network analysis. Using e.g. ideas in [24] the computational
      complexity could be reduced significantly, which could be an
      interesting topic of future research. - Is there any reason
      why the density parameter epsilon is optimized rather than sampled?
      - You might be interested in reading Herlau et al. Modeling
      Temporal Evolution and Multiscale Structure in Networks, ICML 2013,
      which is closely related to your work. I look forward to
      hearing the authors' response to my comments. UPDATE: After
      reading the authors' comments I stand by my assessment. Just to clarify, I
      did not mean to say the use of the birth-death process was wrong, only
      that I find this part of the model unnecessary and not very
      elegant.