Inferring neural population dynamics from multiple partial recordings of the same neural circuit
Simultaneous recordings of the activity of large neural populations are extremely valuable as they can be used to infer the dynamics and interactions of neurons in a local circuit, shedding light on the computations performed. It is now possible to measure the activity of hundreds of neurons using 2-photon calcium imaging. However, many computations are thought to involve circuits consisting of thousands of neurons, such as cortical barrels in rodent somatosensory cortex. Here we contribute a statistical method for "stitching" together sequentially imaged sets of neurons into one model by phrasing the problem as fitting a latent dynamical system with missing observations. This method allows us to substantially expand the population-sizes for which population dynamics can be characterized-beyond the number of simultaneously imaged neurons. In particular, we demonstrate using recordings in mouse somatosensory cortex that this method makes it possible to predict noise correlations between non-simultaneously recorded neuron pairs.

The paper contributes a statistical method to “stitch”
      together sequentially imaged sets of neurons. Such a method expands the
      population sizes for which population dynamics can be characterized beyond
      the number of simultaneously imaged neurons and provide a better image of
      the circuit which cannot be imaged as a whole for experimental and
      technical limitations. The method is then applied on simulated data as
      well as on experimental calcium imaging data in mice. Authors show that
      they can estimate correlations in variability across two populations which
      were not imaged simultaneously. The Authors model the problem as
      fitting a latent dynamical system with missing observations. In this case
      they used simple linear Gaussian system. Four parameters were defined in
      the model: a functional coupling matrix, a stimuli receptive fields’
      matrix which models the stimulus dependence of the population activity and
      two covariance matrices for neural and measurement noise. The parameters
      were estimated from experimental data using a variant of standard EM
      algorithm. Quality The model works nicely on the simulated
      data with two non-overlapping populations of neurons but when applying on
      real data populations must have a certain level of overlap to get
      reasonable results. It is clear why the greater the overlap the better the
      results are, the question is why wasn’t overlap necessary in the simulated
      data example? Also, the authors do not address the source for the
      great difference in correlation shown when applying the model on simulated
      data as in fig 2c and on real data as in fig 3b The beauty of the
      model is the recovery of pairwise correlations between non-simultaneously
      measured neuron pairs. Clarity The paper is well written,
      although all figure legends are not descriptive enough and the text
      related to the figures in the body of the paper does not elaborate enough
      on the figures. Originality The authors list several works
      which have addressed the question of inferring functional connectivity
      from 2-photon imaging data or electrophysiological measurements. However,
      these works do not infer functional connections of non-simultaneously
      imaged neurons. Significance There are many fundamental limits
      to the number of neurons which can be simultaneously imaged. That of
      course, limits our capabilities to reveal the nature of the neural
      computation at the level of the circuit (or more than a few hundreds of
      cells). This paper suggests a method for expanding these limited
      capabilities.


Multi-Prediction Deep Boltzmann Machines
We introduce the multi-prediction deep Boltzmann machine (MP-DBM). The MP-DBM can be seen as a single probabilistic model trained to maximize a variational approximation to the generalized pseudolikelihood, or as a family of recurrent nets that share parameters and approximately solve different inference problems. Prior methods of training DBMs either do not perform well on classification tasks or require an initial learning pass that trains the DBM greedily, one layer at a time. The MP-DBM does not require greedy layerwise pretraining, and outperforms the standard DBM at classification, classification with missing inputs, and mean field prediction tasks.

Deep Boltzmann Machines (DBNs) are usually
      initialized by greedily training a stack of RBMs, and then fine-tuning the
      overall model using persistent contrastive divergence (PCD). To perform
      classification, one typically provides the mean-field features to a
      separate classifier (e.g. a MLP) which is trained discriminatively.
      Therefore the overall process is somewhat ad-hoc, consisting of L + 2
      models (where L is the number of hidden layers) each with its own
      objective. This paper presents a holistic training procedure for DBNs
      which has a single training stage (where both input and output variables
      are predicted) producing models which can classify directly as well as
      efficiently performing other tasks such as imputing missing inputs. The
      main technical contribution is the mechanism by which training is
      performed; a way of training DBNs which uses the mean field equations for
      the DBN to induce recurrent nets that are trained to solve different
      inference tasks (essentially predicting different subsets of observed
      variables). Quality: I think this is a really interesting paper
      and its implications to DBN training are important. I agree with the
      authors that, despite DBNs being really powerful models, the generally
      accepted way of training them has several shortcomings. The authors
      explicitly point these out in section 3: greedily training is sub-optimal,
      using multiple task-specific models loses some benefit of probabilistic
      modeling, and the greedy training procedure is overly complex. The
      motivation is strong, the methods are presented clearly, the work is
      intuitive and clever, and the experiments demonstrate that the method
      works (even if the only datasets considered are the old MNIST and small
      NORB). Clarity: Generally the paper is well written. There are far
      too many top-level sections (11)! Combine 4- 8 into Methods, and 9-10 into
      Experiments. Also, even after reading the caption for figure 2 c) several
      times, I still didn't fully understand it. Originality: To the
      best of my knowledge, the approach is original (an earlier version
      appeared at ICLR workshop track). Significance: The paper is
      important to the deep learning/representation learning community. In fact,
      by simplifying the DBN training procedure, the paper may encourage more
      DBN activity specifically those who don't have experience with layer-wise
      training procedures (as the authors point out in section 1). I wouldn't be
      surprised if this work doesn't lead to more subset-like training
      procedures for other architectures. The multi-inference "trick" described
      in section 5 is interesting in its own right (separate from the DBN
      application). Comments- Even though "y" is almost
      universally used for output and "x" input, the second paragraph in section
      refers to these variables without defining them. Please define these
      variables before using them.- With regards to not being able to
      train a larger MP-DBM on NORB (and thus validate the effectiveness of
      holistic prediction on more than 1 dataset), there is no reason not to try
      distributing the model over multiple GPUs. As well, recent work
      "Predicting Parameters in Deep Learning" (Denil et al. Arxiv) gives
      another way to reduce effective model size but retain capacity without
      having to go to distributed models.


Bayesian inference for low rank spatiotemporal neural receptive fields
The receptive field (RF) of a sensory neuron describes how the neuron integrates sensory stimuli over time and space. In typical experiments with naturalistic or flickering spatiotemporal stimuli, RFs are very high-dimensional, due to the large number of coefficients needed to specify an integration profile across time and space. Estimating these coefficients from small amounts of data poses a variety of challenging statistical and computational problems. Here we address these challenges by developing Bayesian reduced rank regression methods for RF estimation. This corresponds to modeling the RF as a sum of space-time separable (i.e., rank-1) filters. This approach substantially reduces the number of parameters needed to specify the RF, from 1K-10K down to mere 100s in the examples we consider, and confers substantial benefits in statistical power and computational efficiency. We introduce a novel prior over low-rank RFs using the restriction of a matrix normal prior to the manifold of low-rank matrices, and use "localized" row and column covariances to obtain sparse, smooth, localized estimates of the spatial and temporal RF components. We develop two methods for inference in the resulting hierarchical model: (1) a fully Bayesian method using blocked-Gibbs sampling; and (2) a fast, approximate method that employs alternating ascent of conditional marginal likelihoods. We develop these methods for Gaussian and Poisson noise models, and show that low-rank estimates substantially outperform full rank estimates using neural data from retina and V1.

In this paper, the authors develop a sophisticated set
      of Bayesian techniques to infer low-rank neural receptive fields (RFs).
      This is a problem of interest to many modellers and experimenters within
      sensory neuroscience, as identifying the transformations from stimulus
      spaces to output spike trains is both extremely useful for experimental
      protocols, and very difficult or data-intensive due to the very high
      dimensionality of the stimulus space. One option has been to exploit
      potential symmetries in stimulus space, and build low-rank approximations
      to RFs based on sums of space-time (e.g. Pillow et al, 2008) or
      frequency-time (e.g. Ahrens et al 2008) separable kernels. While these
      methods are useful, they have, to date, received less attention from
      model-development front. This has limited their widespread use.
      Here, the authors extend a set of Bayesian inference techniques to
      low-rank RF estimation. This consists of adding hierarchical priors to the
      components of the reduced-rank RFs (using the ALD framework of Park &
      Pillow, 2012), providing full-Bayesian and fast approximate inference
      algorithms for the hierarchical model under a Gaussian likelihood, and
      fast approximate inference method for the popular linear-nonlinear-Poisson
      (LNP) likelihood using a Laplace approximation. The authors then
      demonstrate the utility (and speed) of these methods by fitting low-rank
      RFs to simulated and real neural data from retina and V1. Overall, this
      work is successful and pleasing, generally clear (constrained by the
      required brevity of the format), original, and likely to yield promising
      future results. My only constructive comments on the paper would
      be to clear up a few confusions and add some minor detail: - Lines
      133-141. My understanding of this is that there are separate
      hyperparameters {rho, ..., Phi_f} for C_t and C_x, but this is not clear
      within the text. The current version is written to describe ALD for
      full-rank RFs, and should be amended for the low-rank cases. In the
      sentence "In the ALD prior..." (line 135), the localisation of C_x should
      be in space and spatial frequency, while that for C_t should be in time
      and temporal frequency. Likewise, the description of mu_s and mu_f as
      being centred in space-time and frequency should change (and line 140
      too). Also, mu_s and mu_f as length D vectors, and Phi_s and Phi_f as DxD
      matrices, should be D_x / D_t and D_x by D_x / D_t by D_t respectively,
      with footnote 1 changed to reflect this. - Figs 1-2: space/time
      labelling of axes, as in Fig. 3, would be helpful. - The results
      in Figs 3-4 assume a Gaussian likelihood, yes? I am guessing this due to
      the use of the Gibbs sampler. This should be stated. - Are the
      gains in model performance shown in Figs 3-4 the result of using the
      low-rank approximation, or the result of using the ALD prior? It would be
      helpful to quantify how much is being gained or lost by making the
      low-rank assumption (especially since a major contribution of this paper
      is to extend ALD to the low-rank case). To this end, a comparison against
      a full-rank ALD estimate would be of value: just demonstrating an
      improvement over STA is a straw man. From Fig 5B, my guess would be that
      low-rank ALD would perform the same as full-rank ALD for the Gaussian
      likelihood used in Figs 3-4. Notwithstanding this, the order-of-magnitude
      improvement in compute time for low-rank over full-rank remains useful.
      - As always, the experimenters who would find this work most
      useful would also likely find it near impossible to digest. The net
      utility of this work depends on code being made
      available.


Using multiple samples to learn mixture models
In the mixture models problem it is assumed that there are K distributions θ 1 , . . . , θ K and one gets to observe a sample from a mixture of these distributions with unknown coefficients. The goal is to associate instances with their generating distributions, or to identify the parameters of the hidden distributions. In this work we make the assumption that we have access to several samples drawn from the same K underlying distributions, but with different mixing weights. As with topic modeling, having multiple samples is often a reasonable assumption. Instead of pooling the data into one sample, we prove that it is possible to use the differences between the samples to better recover the underlying structure. We present algorithms that recover the underlying structure under milder assumptions than the current state of art when either the dimensionality or the separation is high. The methods, when applied to topic modeling, allow generalization to words not present in the training data.

The problem of mixture model learning is considered,
      assuming the availability of samples from the same mixture components, but
      with different mixing weights. Two algorithms are described and analyzed.
      The first, Multi Sample Projection (MSP), is a dimensionality
      reduction algorithm that seeks to find the subspace spanned by the means
      of the mixture components. MSP projects the data on the subspace spanned
      by the overall means of each of the available sets of samples. The
      theoretical analysis shows that the resulting procedure will maintain the
      separation between the components assuming the dimension isn't too large,
      and that there is sufficient variation in the mixing weight of each
      mixture component between the samples. The second, Double Sample
      Clustering (DSC), assumes that exactly two sets of samples are available,
      that the mixture components have disjoint support, and that the
      (appropriately normalized) mixture weights for any subset of the
      components differs by some minimal amount between the two mixtures. The
      algorithm iteratively trains binary classifiers on reweighted subsets of
      the data sets, the intuition being that the support sets of the mixture
      components are regions where the sign of the difference of the densities
      of the two mixtures are constant. The general idea of taking
      advantage of the "multiple sample" setting for clustering (there has to be
      a less ambiguous name for that by the way, but I can't think of one...) is
      interesting, and this paper may lead to further development of algorithms
      and theory on the topic. The theoretical analysis of MSP and DSC
      are not much stronger than showing conditions for consistency, which is
      all that can be expected without further assumptions. As the authors point
      out, sharper rates could be proven for the preservation of distances in
      MSP if, for instance, a Gaussian mixture was assumed. The
      numerical experiments provide encouraging evidence for the practical
      benefit of MSP and DSC in the multiple sample setting. Small
      issues: It is not clear to the reviewer why there is an exact
      equality in part 2 of Lemma 1, as opposed to a \leq. Definition 1
      is a bit confusing. My understanding is that here L refers to a base
      classifier that will be used in the iterations of the algorithm, but the
      conditions of the definition seem to be overly general (it allows any
      measure over X and +/-1, for one thing, which doesn't have anything to do
      with C_1,..., C_K; also condition 3 brings in C_I, even though, again, the
      input to the algorithm may have nothing to do with the clusters). Perhaps
      it would be clearer if Assumption 4 from section A.3 was used instead.
      Also, there is no mention of what delta is; I'm guessing there should be a
      "with probability 1-delta" somewhere, as well as an explanation that n is
      the number of samples to be used by the algorithm L (n was used for the
      number of dimensions earlier in the paper).


Robust learning of low-dimensional dynamics from large neural ensembles
Recordings from large populations of neurons make it possible to search for hypothesized low-dimensional dynamics. Finding these dynamics requires models that take into account biophysical constraints and can be fit efficiently and robustly. Here, we present an approach to dimensionality reduction for neural data that is convex, does not make strong assumptions about dynamics, does not require averaging over many trials and is extensible to more complex statistical models that combine local and global influences. The results can be combined with spectral methods to learn dynamical systems models. The basic method extends PCA to the exponential family using nuclear norm minimization. We evaluate the effectiveness of this method using an exact decomposition of the Bregman divergence that is analogous to variance explained for PCA. We show on model data that the parameters of latent linear dynamical systems can be recovered, and that even if the dynamics are not stationary we can still recover the true latent subspace. We also demonstrate an extension of nuclear norm minimization that can separate sparse local connections from global latent dynamics. Finally, we demonstrate improved prediction on real neural data from monkey motor cortex compared to fitting linear dynamical models without nuclear norm smoothing.

The authors present a method for reducing the
      dimensionality of neural activity data. They take advantage of recent
      advances in applied mathematics to solve sparse and low-rank matrix
      approximation problems using convex relaxation, as well as of Bregman
      divergences for exponential families. They apply the method to synthetic
      data. I find the work mathematically sound and the presentation
      generally clear. However, the authors may want to devote more attention to
      introducing the problem. Indeed, in the beginning of Section 2, they
      postulate the existence of a low-dimensional subspace of neuronal
      activity. It would seem to me that the dynamics in this low-dimensional
      subspace, Eq.(1), should be mediated by neurons and their synaptic
      connections. However, Eq.(1) does not include the synaptic connectivity
      matrix D, which instead appears in Eq.2. Because the authors point out
      that the low-dimensional dynamics would be present even in the absence of
      inputs (u_t = 0) I don’t understand what physical substrate underlies the
      dynamics of low-dimensional activity. Additionally, while I admire
      the authors for their effort to introduce fashionable applied mathematics
      techniques into neuroscience, it is difficult to evaluate the value of
      their method until it is applied to actual data and generates insight into
      the function of a biological system. Minor on line 291:
      minimizing D -> minimizing over D


Estimation Bias in Multi-Armed Bandit Algorithms for Search Advertising
In search advertising, the search engine needs to select the most profitable advertisements to display, which can be formulated as an instance of online learning with partial feedback, also known as the stochastic multi-armed bandit (MAB) problem. In this paper, we show that the naive application of MAB algorithms to search advertising for advertisement selection will produce sample selection bias that harms the search engine by decreasing expected revenue and "estimation of the largest mean" (ELM) bias that harms the advertisers by increasing game-theoretic player-regret. We then propose simple bias-correction methods with benefits to both the search engine and the advertisers.

This paper explores the task of search advertising
      auctions. In a second-price auction, the auctioneer must consider both the
      per-click price offered by the advertisers (which is known) and the
      probability that users will click the ad (which must be estimated from
      data). Multi-armed bandit algorithms can be used to choose which ads to
      run when estimating the click probability for each ad, since it is
      critical to have accurate estimates of both the highest and second-highest
      value advertisers in order to run the auction. The authors
      identify and address two sources of bias that affect this task, impacting
      both the auctioneer and advertisers' profits. First, they find that
      multi-armed bandit algorithms tend to underestimate the value being
      estimated due to sample selection bias, as overestimated values are
      resampled (and corrected) while underestimated values receive fewer
      samples (and remain underestimates). In a second-price auction, the
      auctioneer then charges a lower price than they could on expectation if
      the true click probabilities were known. Bounds on this loss are proven,
      and the auctioneer's loss does not converge to zero due to the sample
      selection bias. They give an example of a simple method for correcting
      this bias. Second, they find that advertisers in this auction are not
      acting optimally by bidding their true value, as is sought with a
      second-price auction; instead, they can increase their expected value by
      under-bidding while still bidding enough that their ads are bought. The
      auctioneer will tend to overestimate the second price used for the
      auction, even if its estimates for each individual advertiser are
      unbiased. By underbidding an agent can cause the auctioneer to obtain
      better estimates of the other advertisers' rates, thus lowering the price
      that the winner will be charged. While this bias cannot be provably
      corrected for, the authors show a technique for reducing it. Empirical
      results are presented that demonstrate both sources of bias, and how their
      correction measures decrease auctioneer loss and incentivize advertisers
      to bid closer to their true value. QUALITY: The paper makes a
      good contribution. It demonstrates sources of bias in the search
      advertising task, proves bounds on them, presents corrections or
      reductions, and demonstrates the effects empirically. CLARITY:
      The paper is well written and easy to follow. ORIGINALITY:
      I'm far enough from this domain that it's difficult to judge
      originality. The sources of bias highlighted in the paper appear novel, as
      do the bounds and solutions. SIGNIFICANCE: The paper
      demonstrates two sources of bias in the search advertising task, corrects
      them, and demonstrates the resulting improvement in utility for the
      auctioneer. This should be a useful result to consider in future work in
      this domain.


Binary to Bushy: Bayesian Hierarchical Clustering with the Beta Coalescent
Discovering hierarchical regularities in data is a key problem in interacting with large datasets, modeling cognition, and encoding knowledge. A previous Bayesian solution-Kingman's coalescent-provides a probabilistic model for data represented as a binary tree. Unfortunately, this is inappropriate for data better described by bushier trees. We generalize an existing belief propagation framework of Kingman's coalescent to the beta coalescent, which models a wider range of tree structures. Because of the complex combinatorial search over possible structures, we develop new sampling schemes using sequential Monte Carlo and Dirichlet process mixture models, which render inference efficient and tractable. We present results on synthetic and real data that show the beta coalescent outperforms Kingman's coalescent and is qualitatively better at capturing data in bushy hierarchies.

This paper develops a model for multifurcating trees
      with edge lengths and observed data at the tree leaves; the model is based
      on the beta coalescent from the probability literature. The authors
      develop an MCMC inference scheme for their model, in which they draw on
      existing work that uses belief propagation to perform inference for the
      Kingman coalescent (an edge case of the beta coalescent in which all trees
      are binary). The particular challenge for inference here is that there are
      many more possible parent-child node relationships when parents can have
      multiple children (not just two). The authors seem to use a Dirichlet
      Process mixture model (DPMM) at each node to narrow down the space of
      possible children subsets to consider. As the authors note, even inference
      with the Kingman coalescent is a hard problem. In experiments, they
      compare to the Kingman coalescent and hierarchical agglomerative
      clustering. The Kingman coalescent is a popular modeling tool, so
      it is great to see a practical extension of the Kingman coalescent to the
      multifurcating case being explored for inference. The authors give a
      convincing discussion of the interest in multifurcating trees. The DPMM
      trick to narrow the space of children to consider for a parent is another
      neat idea here and seems to be validated experimentally. That being said,
      there seem to be some issues with readability and notation in the paper
      (in particular, it is difficult to read on its own without the Teh, Daume
      III, Roy paper and also its own supplementary material, and even then
      there are some confusing points). And while I think that it is useful to
      have a beta coalescent extension to the Kingman coalescent regardless, it
      seems surprising that the Knowles, Ghahramani 2011 work on multifurcating
      trees (which comes from the diffusion, rather than coalescing, perspective
      for building trees) is not included in the experimental comparison (or at
      least discussed more). Moreover, given that the main selling point here
      seems to be the use of multifurcating trees in modeling, it would be
      useful to see comparisons of the trees returned by the Kingman and beta
      coalescent approaches. Some further major points: * Page 2,
      lines 56-57: My understanding is that this sentence is saying that the
      Dirichlet diffusion tree (DDT) and Pitman-Yor diffusion tree (PYDT) do not
      include a generative model for edge lengths. Such a statement would be
      false. An integral component of both models is the distance between child
      and parent nodes, which is determined by a divergence function. These
      trees also typically come with a (Brownian-motion-derived) generative
      model for leaf node locations in some (typically Euclidean) space. That
      the DDT and Kingman coalescent are in some sense inverses has been
      exploited for modeling (see the recent work of Teh et al on
      fragmentation-coagulation processes). From this perspective, it seems
      disingenuous to claim that this paper is the first to provide a model of
      edge lengths for multifurcating trees since the PYDT does so. The
      contributions of the current work seem, rather, to be the development of
      inference with the beta coalescent; these presumably have their own merits
      over, and separate from, the PYDT (just as the Kingman coalescent and DDT
      have their own merits). * A number of points in Algorithm 1 are
      unclear. ---- * Step 8: Is i local to s? Should it have an s
      subscript? Or should this incrementation be outside the for loop? ----
      * Step 14: pi doesn't really seem to be defined in the main text. Page 2,
      line 67: The "tree structure" pi is mentioned but not defined, and Page 4,
      line 166: the "tree structure" theta_i is mentioned but not defined. How
      does pi differ from theta? Without definition, it's unclear what the
      subtraction and addition in Step 14 mean. * There are other notational
      issues: ---- * It is unclear what the role of p_0(y_i) is or how it is
      chosen. It is not clear why the current authors diverge from Teh et al,
      who seem to use a base distribution at the root, and here instead use a
      distribution at each node. How does this distribution interact with the
      transition kernel? Z_{-\infty}(x) does not seem to be defined explicitly
      (Eq. 6). ---- * The superscript s seems to be introduced on page 4,
      line 168 without being defined or discussed. There later seems to be some
      confusion about what is particle-specific and what is not. E.g., line 241,
      page 5: what does Omega_i restrict? All particles? The notation for
      Omega_i later switches to an s dependence. If a DPMM is run for each (s,i)
      pair, it would be helpful to be explicit in the text about this.
      Conversely, in Eq. 11, why does lambda_{n_i - 1}^s have an s superscript?
      That seems to at least overload the notation on page 2, line 83. *
      Some details and choices in the experiments were confusing, e.g.: ----
      * Why was alpha = 1.2 chosen for the beta coalescent? This seems somewhat
      arbitrary. Likewise, why the choice of 5 chains with 5 particles per chain
      (page 7, line 364) vs 5 chains and 80 particles each later (page 8, line
      401)? Also on line 364, how many dimensions were kept after using PCA?
      ---- * In the definition of the purity score, is the purity score
      averaged over all pairs? ---- * Page 7, line 338 suggests that the
      synthetic data comes from a beta coalescent generative process, but the
      appendix seems to demonstrate this is not true. That the generating
      process is not a beta coalescent should be clearer in the main text.
      Moreover, in the supplementary material, it's not clear how the actual
      nodes (as opposed to the number of nodes) for merging are chosen at each
      round. Presumably the true model is not used for synthetic data generation
      because it is too computationally complex; this is worth mentioning in the
      text. ---- * It's unclear why the effects described on line 345 on
      page 7 are evident. Does increasing the number of observations have no
      effect simply because the maximum (and best) score is already attained?
      ---- * How were the sample hierarchies in Figs. 3 and 4 chosen? Were
      they, e.g., just the final samples in the chain? Minor comments:
      * Page 1, line 38 (and elsewhere): The term "bushy" is often used to
      describe binary trees (in particular, balanced binary trees). The authors
      here seem to use it describe multifurcating-ness of trees. If the authors
      wish to use the word "bushy", I would suggest defining their desired usage
      early on (at least before the first use in the main text besides section
      headings). * Page 2, line 80: Be explicit about the Kingman coalescent
      representation as a Lambda coalescent. Namely, the measure Lambda is the
      Dirac delta measure at 0 in this case. * Page 2, line 103: Can
      anything specific be said to support the statement that this approach is
      "more scalable"? E.g., some asymptotic bounds/analysis? * Page 4, line
      187: Is this meant to be computational complexity just for the ith
      coalescent event? If so, n_i - 1 or n_i should be used instead of n. If
      it's for all events, since the Kingman coalescent considers every pair of
      nodes after each coalescent event in the tree, it would seem the cost of
      growing a Kingman coalescent is something like a constant times n^2 +
      (n-1)^2 + (n-2)^2 + ... + , which would be asymptotically like n^3, not
      n^2. And the result for the beta coalescent would be not obvious in this
      case. * Page 5, line 217: Consider using a different notation for the
      identity matrix and the indicator function (cf. Eq. 9). Define the
      identity matrix notation at first use (here) instead of line 92, page 2 in
      the supplementary material. I'm not sure why ref [10] is referenced on
      page 5, line 217 or why this relationship between the mutation rate and
      Dirichlet concentration parameter is chosen. * Eq. 12: Should this be
      t_b instead of t_{b_i}? * In supplementary material Section 2, x_n is
      overloading notation with the main paper. Also, n_k should be explicitly
      defined, and also z_i^k (which appears to be 1\{ z_i == k\} ). ===
      Update after Author Feedback === The Author Feedback addressed my
      concerns as much as seems possible in the constrained space. I am
      optimistic this will be a solid paper after a round of
      editing.


Reward Mapping for Transfer in Long-Lived Agents
We consider how to transfer knowledge from previous tasks (MDPs) to a current task in long-lived and bounded agents that must solve a sequence of tasks over a finite lifetime. A novel aspect of our transfer approach is that we reuse reward functions. While this may seem counterintuitive, we build on the insight of recent work on the optimal rewards problem that guiding an agent's behavior with reward functions other than the task-specifying reward function can help overcome computational bounds of the agent. Specifically, we use good guidance reward functions learned on previous tasks in the sequence to incrementally train a reward mapping function that maps task-specifying reward functions into good initial guidance reward functions for subsequent tasks. We demonstrate that our approach can substantially improve the agent's performance relative to other approaches, including an approach that transfers policies.

This paper presents a model for transfer where
      synthetic reward functions are learned from previous tasks, and are mapped
      to new tasks to produce synthetic reward functions which speed up
      learning. This application of the relatively recent idea of a synthetic
      reward function that improves the performance of an underlying "utility"
      reward function to the transfer scenario is new and interesting, and the
      experiments are well thought out, thorough and reasonably convincing.
      The major criticism I had of the paper is that it does not compare
      to other methods of transfer. The authors are incorrect that policy
      transfer is not possible in their setting; that is why sub-policies, in
      the form of options, are often transferred instead of entire policies.
      (Speaking of which, the authors should probably cite Lisa Torrey's work on
      policy transfer.) In this case though it's OK to skip that comparison
      because you could use either, both, or neither types of transfer in
      combination, so the comparison adds information but is not critical.
      However, the failure to compare against a similar reward shaping
      scheme is more problematic. These two methods are effectively solving the
      same problem, except that reward shaping does not change the ultimate
      solution and a synthetic reward function does. In my opinion, these two
      methods (even outside of the transfer scenario) have not been adequately
      compared, which is odd because its trivial to learn a shaping function
      (it's just a value function initialization, so you're just learning a
      value function). This lack of comparison leaves me with significant doubts
      about the whole synthetic reward function enterprise generally. So I think
      a comparison here - where the shaping function is mapped in the same way
      that the reward function is - would significantly improve the paper. But
      perhaps that is too much to ask for in a single paper, especially since
      the mapped shaping function could be considered a new (though somewhat
      obvious) method. The paper is very well written and was generally
      a pleasure to read, though this was spoiled somewhat by the repeated use
      of parenthetical citations as nouns. The references were poorly formatted
      in some cases (capitalization on "mdps", etc.)


DeViSE: A Deep Visual-Semantic Embedding Model
Modern visual recognition systems are often limited in their ability to scale to large numbers of object categories. This limitation is in part due to the increasing difficulty of acquiring sufficient training data in the form of labeled images as the number of object categories grows. One remedy is to leverage data from other sources -such as text data -both to train visual models and to constrain their predictions. In this paper we present a new deep visual-semantic embedding model trained to identify visual objects using both labeled image data as well as semantic information gleaned from unannotated text. We demonstrate that this model matches state-of-the-art performance on the 1000-class ImageNet object recognition challenge while making more semantically reasonable errors, and also show that the semantic information can be exploited to make predictions about tens of thousands of image labels not observed during training. Semantic knowledge improves such zero-shot predictions achieving hit rates of up to 18% across thousands of novel labels never seen by the visual model.

Summary of paper: This computer vision paper uses an
      unsupervised, neural net based semantic embedding of a Wikipaedia text
      corpus trained using skip-gram coding to enhance the performance of the
      Krizhevsky et al deep network [11] that won the 2012 ImageNet large scale
      visual recognition challenge, particularly for zero-shot learning problems
      (i.e. previously unseen classes with some similarity to previously seen
      ones). The two networks are trained separately, then the output layer of
      [11] is replaced with a linear mapping to the semantic text representation
      and re-trained on ImageNet 1k using a dot product loss reminiscent of a
      structured output SVM one. The text representation is not currently
      re-trained. The model is tested on ImageNet 1k and 21k. With the semantic
      embedding output it does not quite manage to reproduce the ImageNet 1k
      flat-class hit rates of the original softmax-output model, but it does
      better than the original on hierarchical-class hit rates and on previously
      unseen classes from ImageNet 21k. For unseen classes, the improvements are
      modest in absolute terms (albeit somewhat larger in relative ones).
      Review: I think this is an interesting paper that can be accepted.
      The subject area (scaling to very large numbers of visual classes,
      combining modalities, deep learning) is clearly topical. This is not the
      first method to combine visual and textual information, but AFAIK it is
      the first to tackle this problem at such a large scale (all 21k ImageNet
      classes), especially using a fully unsupervised text model. The absolute
      improvements in recognition rates are fairly modest, but this is a
      challenging area where any improvement is welcome. Further points
      I have several questions for the authors: - The skip-gram
      textual model is quite weak in the sense that no prior semantics is built
      in. It seems plausible that strengthening the semantic cues (e.g. with a
      WordNet distance based regularizer) would improve the embedding. Keying on
      Wikipaedia structures might also help, e.g. a "blue shark" might have its
      own page, with links back to the generic "shark" page. - It would
      be useful to quantify the improvements available by refining the textual
      embedding during the joint training phase. - How do adjectives /
      visual attributes come into this? Many classes have descriptive names,
      e.g. one might expect a "whitecap shark" to have some body part with a
      white cap. It is not clear to me that the current embedding model exploits
      such hints / such factorization in any usable way (e.g. as a "white" basis
      vector to be mixed in to the "shark" dot product). - What are the
      flat precision scores for zero-shot DeVISE? - I ask because the
      hierarchical metric necessarily confuses the issue, especially at such low
      absolute precisions. E.g. if images of blue sharks are classified mainly
      as sharks, but not particularly as blue ones, the hierarchical score could
      still be quite high but one would hesitate to claim that a blue shark
      classifier has been learned. I suspect that this is happening here, at
      least to some extent. Note added after rebuttal: Excuse me
      for my garbled question about flat hit@k precision scores for zero shot
      DeVISE. I realise that you have already given these. What I meant to ask
      for was scores for a *hierarchical version* of the flat hit@k metric. Flat
      hit@k is the familiar "best of k" metric -- only the best of the k guesses
      counts towards the score. Your hierarchical precision@k is interesting but
      very different: it essentially measures how well the full set of k guesses
      reproduces the local categorical neighbourhood of the one true class. As
      such it is less robust than a "best of k" metric: even if a method
      invariably determines the true class with one of its k guesses, it will
      score badly on hierarchical precision@k if its other k-1 guesses are
      typically far from the true class. Given that you report the two metrics
      face to face and don't fully explain hierarchical precision@k in the
      paper, unwary readers are likely to think that hierarchical precision@k is
      a hierarchical "best of k" metric, and hence be mislead by the results.
      Also, to get a handle on the types of errors that are being made, it would
      be convenient to report at least some results in a metric that supports a
      "hops from ground truth" score decompositon. E.g. with the hierarchical
      best of k metric that counts how many hops from the true class the best
      [*] of the k guesses is, you can report accuracies as (say) 10% hits at
      zero hops (the flat hit@k score) plus 5% hits at one hop plus ... If
      desired this can be refined by breaking hop ties, e.g. counting children
      as closer than parents you might get "plus 3% hits to a child class plus
      2% hits to the parent class plus...". [*] Only one of the k
      guesses ever counts. Ties are broken arbitrarily. Further points:
      - Please use a uniform reporting convention for all performance
      score tables -- convert either Table 3 to percentages, or Table 2 to
      probabilities.


Online Variational Approximations to non-Exponential Family Change Point Models: With Application to Radar Tracking
The Bayesian online change point detection (BOCPD) algorithm provides an efficient way to do exact inference when the parameters of an underlying model may suddenly change over time. BOCPD requires computation of the underlying model's posterior predictives, which can only be computed online in O(1) time and memory for exponential family models. We develop variational approximations to the posterior on change point times (formulated as run lengths) for efficient inference when the underlying model is not in the exponential family, and does not have tractable posterior predictive distributions. In doing so, we develop improvements to online variational inference. We apply our methodology to a tracking problem using radar data with a signal-to-noise feature that is Rice distributed. We also develop a variational method for inferring the parameters of the (non-exponential family) Rice distribution.
Change point detection has been applied to many applications [5; 7]. In recent years there have been great improvements to the Bayesian approaches via the Bayesian online change point detection algorithm (BOCPD) [1; 23; 27]. Likewise, the radar tracking community has been improving in its use of feature-aided tracking [10]: methods that use auxiliary information from radar returns such as signal-to-noise ratio (SNR), which depend on radar cross sections (RCS) [21]. Older systems would often filter only noisy position (and perhaps Doppler) measurements while newer systems use more information to improve performance. We use BOCPD for modeling the RCS feature. Whereas BOCPD inference could be done exactly when finding change points in conjugate exponential family models the physics of RCS measurements often causes them to be distributed in non-exponential family ways, often following a Rice distribution. To do inference efficiently we call upon variational Bayes (VB) to find approximate posterior (predictive) distributions. Furthermore, the nature of both BOCPD and tracking require the use of online updating. We improve upon the existing and limited approaches to online VB [24; 13]. This paper produces contributions to, and builds upon background from, three independent areas: change point detection, variational Bayes, and radar tracking.
Although the emphasis in machine learning is on filtering, a substantial part of tracking with radar data involves data association, illustrated in Figure 1. Observations of radar returns contain measurements from multiple objects (targets) in the sky. If we knew which radar return corresponded to which target we would be presented with N T ∈ N 0 independent filtering problems; Kalman filters [14] (or their nonlinear extensions) are applied to "average out" the kinematic errors in the measurements (typically positions) using the measurements associated with each target. The data association problem is to determine which measurement goes to which track. In the classical setup, once a particular measurement is associated with a certain target, that measurement is plugged into the filter for that target as if we knew with certainty it was the correct assignment. The association algorithms, in effect, find the maximum a posteriori (MAP) estimate on the measurement-to-track association. However, approaches such as the joint probabilistic data association (JPDA) filter [2] and the probability hypothesis density (PHD) filter [16] have deviated from this.
To find the MAP estimate a log likelihood of the data under each possible assignment vector a must be computed. These are then used to construct cost matrices that reduce the assignment problem to a particular kind of optimization problem (the details of which are beyond the scope of this paper). The motivation behind feature-aided tracking is that additional features increase the probability that the MAP measurement-to-track assignment is correct. Based on physical arguments the RCS feature (SNR) is often Rice distributed [21,Ch. 3]; although, in certain situations RCS is exponential or gamma distributed [26]. The parameters of the RCS distribution are determined by factors such as the shape of the aircraft facing the radar sensor. Given that different aircraft have different RCS characteristics, if one attempts to create a continuous track estimating the path of an aircraft, RCS features may help distinguish one aircraft from another if they cross paths or come near one another, for example. RCS also helps distinguish genuine aircraft returns from clutter: a flock of birds or random electrical noise, for example. However, the parameters of the RCS distributions may also change for the same aircraft due to a change in angle or ground conditions. These must be taken into account for accurate association. Providing good predictions in light of a possible sudden change in the parameters of a time series is "right up the alley" of BOCPD and change point methods.
The original BOCPD papers [1; 11] studied sudden changes in the parameters of exponential family models for time series. In this paper, we expand the set of applications of BOCPD to radar SNR data which often has the same change point structure found in other applications, and requires online predictions. The BOCPD model is highly modular in that it looks for changes in the parameters of any underlying process model (UPM). The UPM merely needs to provide posterior predictive probabilities, the UPM can otherwise be a "black box." The BOCPD queries the UPM for a prediction of the next data point under each possible run length, the number of points since the last change point. If (and only if by Hipp [12]) the UPM is exponential family (with a conjugate prior) the posterior is computed by accumulating the sufficient statistics since the last potential change point. This allows for O(1) UPM updates in both computation and memory as the run length increases. We motivate the use of VB for implementing UPMs when the data within a regime is believed to follow a distribution that is not exponential family. The methods presented in this paper can be used to find variational run length posteriors for general non-exponential family UPMs in addition to the Rice distribution. Additionally, the methods for improving online updating in VB (Section 2.2) are applicable in areas outside of change point detection. To our knowledge this paper is the first to demonstrate how to compute Bayesian posterior distributions on the parameters of a Rice distribution; the closest work would be Lauwers et al. [15], which computes a MAP estimate. Other novel factors of this paper include: demonstrating the usefulness (and advantages over existing techniques) of change point detection for RCS estimation and tracking; and applying variational inference for UPMs where analytic posterior predictives are not possible. This paper provides four main technical contributions: 1) VB inference for inferring the parameters of a Rice distribution. 2) General improvements to online VB (which is then applied to updating the UPM in BOCPD). 3) Derive a VB approximation to the run length posterior when the UPM posterior predictive is intractable. 4) Handle censored measurements (particularly for a Rice distribution) in VB. This is key for processing missed detections in data association.

This is an interesting and novel paper about using
      variational inference in Bayesian online change point detection. It
      seemed to me that the experimental results are interesting and
      thorough, though I am not in the field of radar tracking. A
      problem with the paper is that I found the exposition hard to
      follow---a more holistic view of the algorithm is needed to make the
      paper clearer. (That is, after reading the paper, I'm not sure I could
      implement it.) Perhaps you can include pseudocode with pointers to
      various equations. Comments: p3: Intuitions about the
      Rice distribution would be nice. When and why is it used in tracking?
      p3: Nitpick: Variational Bayes finds a *local* optimum.
      p4: Variational inference is easiest in the settings you describe,
      those that are conjugate-exponential with latent variables. But see
      recent work like [1] and [2] which seek to expand on this. p7:
      I was intrigued by the comment "improved online updating in terms of
      KL to batch" but could not unpack it---I don't think that turn of
      phrase appears earlier to explain an idea in the algorithm. Here again
      a holistic summary of the algorithm would help make the paper clearer.
      p8: In the conclusions, "closing a hole" is a little strong. You
      needed a variational approximation to the Rice so you developed it.
      This is straightforward because it is a conjguate-exponential model.
      (This is not to take away from the contribution---you make a good case
      via an interesting application for the Rice being a distribution for
      which we want a variational approximation.) Also, I found
      referring to "non-exponential family distributions" confusing. Yes,
      the Rice is not in the exponential family but, because I read many
      papers about variational inference, I read the expression to mean
      latent variable models that are not in the conjugate-exponential
      family, which the Rice is in. (No need to change if you feel
      otherwise.)


Demixing odors -fast inference in olfaction
The olfactory system faces a difficult inference problem: it has to determine what odors are present based on the distributed activation of its receptor neurons. Here we derive neural implementations of two approximate inference algorithms that could be used by the brain. One is a variational algorithm (which builds on the work of Beck. et al., 2012), the other is based on sampling. Importantly, we use a more realistic prior distribution over odors than has been used in the past: we use a "spike and slab" prior, for which most odors have zero concentration. After mapping the two algorithms onto neural dynamics, we find that both can infer correct odors in less than 100 ms. Thus, at the behavioral level, the two algorithms make very similar predictions. However, they make different assumptions about connectivity and neural computations, and make different predictions about neural activity. Thus, they should be distinguishable experimentally. If so, that would provide insight into the mechanisms employed by the olfactory system, and, because the two algorithms use very different coding strategies, that would also provide insight into how networks represent probabilities.

Comments: This paper describes two candidate
      mechanisms for implementing probabilistic inference in the olfactory
      system, one based on variational inference and a second based on
      sampling. The basic problem is to infer the (marginal probability
      over) the identity and concentration of odors present in a sparse
      mixture from the noisy responses of a population of olfactory receptor
      neurons. The ORNs respond linearly with Poisson noise, and the
      proposed algorithms perform inference under a spike-and-slab prior on
      odors. The problem is clearly motivated and the work itself is
      very timely given the recent surge of interest in mechanisms for
      performing Bayesian inference in neural circuits (and in the olfactory
      system in particular). The ideas are novel and the performance seems
      impressive; I'm surprised by the improvement over template matching.
      This is certainly one of the more creative and more interesting papers
      I reviewed this year. The paper seems well suited to NIPS
      insofar as it brings together theoretical ideas about algorithms for
      probabilistic inference and practical ideas about how to implement
      them in neural hardware (a consideration that does not arise in
      standard machine learning papers). Comments: -
      Variational algorithm: the intuition for the steps in deriving the
      variational approximation (pg 3) are not exceedingly clear, and some
      of the assumptions seem like they might be rather severe. In light of
      this, it seems surprising that the variational algorithm works so
      well, that is, that the sampling-based estimate isn't more accurate.
      - Sampling algorithm: some clever ideas here. I like the trick for
      changing variables to \tilde c_j and the idea for asynchronous Gibbs
      sampling. However, the relationship between s' and \tilde c (the
      variables being sampled) and neural activity in the network should
      be spelled out more clearly. - Concern about plausibility: the
      inference algorithms both work in real time, but the ORN responses r_i
      seem to be fixed (discrete) spike counts, which doesn't seem well
      matched. Can the same algorithms be applied with point process ORN
      responses? Also, it would help to include some detail about how
      realistic the assumptions about connectivity and number of ORNs is
      (for simulations described in Section 4). - It would be nice
      to have some more explicit predictions about the form of the neural
      activity itself (i.e., as opposed to predictions about the
      relationship between neural activity and probability). The prediction
      of "increased variability" for sampling-based inference doesn't seem
      very specific, since one could also implement a dynamical system for
      variational inference in which the individual responses are very
      noisy. Are there any data currently in existence that might be
      compared to model predictions? Minor comments: - eq 2.2a:
      subscripts on alpha and Beta seem slightly puzzling, especially since
      they're missing in eq. 2.3. (It's not till we see the variational
      approximation section that we see why they need to be distinguished
      from alpha_0 and Beta_0). - eq 2.2b: should say what \pi is
      - pg 2, line 098: not clear what it means that alpha_0 and and
      Beta_0 can model the effects of the baseline firing rate. - pg
      2: clever use of multinomial theorem, but the N_ij need clearer
      definition. The "N_ij is shorthand for..." sentence is very
      confusing. Indices of summation (j=1 to to r_j?) should be given in eq
      3.2. - I wonder if there's any particular reason to use
      Langevin sampling (as opposed to either vanilla Metropolis-Hastings or
      something fancier).


Reasoning With Neural Tensor Networks for Knowledge Base Completion
Knowledge bases are an important resource for question answering and other tasks but often suffer from incompleteness and lack of ability to reason over their discrete entities and relationships. In this paper we introduce an expressive neural tensor network suitable for reasoning over relationships between two entities. Previous work represented entities as either discrete atomic units or with a single entity vector representation. We show that performance can be improved when entities are represented as an average of their constituting word vectors. This allows sharing of statistical strength between, for instance, facts involving the "Sumatran tiger" and "Bengal tiger." Lastly, we demonstrate that all models improve when these word vectors are initialized with vectors learned from unsupervised large corpora. We assess the model by considering the problem of predicting additional true relations between entities given a subset of the knowledge base. Our model outperforms previous models and can classify unseen relationships in WordNet and FreeBase with an accuracy of 86.2% and 90.0%, respectively.

This paper describes a model for multi-relational data
      specialized to knowledge bases and other datasets with entities and
      relationships labeled by words. The model is a straightforward
      generalization of other models from the literature. The model uses entity
      vectors that are the average of word vectors in the entity name. The score
      of a triple (e1, R, e2) is based on three vector-valued terms fed through
      an elementwise nonlinearity and then dotted with a relation-dependent
      vector. One term is a bias vector that is a function of R. One term has a
      weight matrix dependent on R applied to the concatenation of the vectors
      for e1 and e2. The final term is a bilinear tensor product using a
      relation-dependent set of weights and the two entity vectors. To justify
      the use of this model, the paper presents results on classifying (entity,
      relation, entity) triples as positive (from a knowledge base) or negative
      (synthetic triples unlikely to exist in the knowledge base). I
      don't see the advantage of the non-linearity in equation 1. I expect the
      model would be mostly unchanged in practice if f was removed from equation
      1 (and u_R was folded into other parameters where appropriate). When all
      the embedding vectors are learned and unconstrained and there are so many
      arbitrary relation-dependent weights the nonlinearity adds very little
      power to the model since the mapping from entity id to entity vector is
      arbitrary. Although the model in the paper is powerful and
      potentially interesting, the experimental evaluation has a few serious
      weaknesses. The full train/dev/test splitting protocal needs to be
      described precisely. If (x,r,y) is in the test set, can (x,r, z != y) be
      in the training set? Or can only (u!=x, r, v!=y) and (x, q != r, y) be in
      the training set? The most powerful models should do best when they have
      lots of data and have seen entities in the test set many times and seen
      relations in the test set many times. Therefore the most interesting
      experiments for the model described in the paper are ones that test the
      generalization ability of the model from limited training data. Along
      these lines, why are results on all relations in Freebase relegated to the
      supplementary material and so few relations used in the main paper? A
      dozen relations is simply not enough to do any meaningful evaluation of
      this model, especially since it has so many relation-specific parameters.
      And how many entities are there for the full experiments? If anything,
      remove the experiments that focus on the small subset of Freebase and if
      necessary break down some of the results based on how much relevant
      training data was used for the particular test cases. The model needs to
      be tested in the regime where it has very little data for the particular
      relation being tested and in the regime where it has very little data for
      the entities; that is where simpler special cases of the model may shine.
      My own view is that a simpler model will probably work better in most
      situations although it is possible we as a community have not yet settled
      on the best simple model. There are many obvious extensions to models in
      the literature that add modeling power and when a paper introduces one it
      must demonstrate that the overfitting risks can be controlled and that the
      new model is actually really useful. Figure 3 demonstrates what is
      probably an overfitting effect for the NTN model on WordNet since the NTN
      does so poorly when trained without word vectors. This supports my concern
      that overfitting is a larger than usual risk with this model and that the
      best evaluation would employ data with relations that have few training
      cases associated with them. The explanation of how negative cases
      were generated for the classification task needs more detail. Can the
      negative triple (Pablo Picasso, nationality, George Washington) get
      generated? I don't understand why a classification task was used
      instead of a ranking or information retrieval style task as in [7]. Using
      a classification task seems non-standard and makes it harder to compare to
      other papers. Also for open domain knowledge base completion, a ranking
      task makes more sense. I recommend removing the average column of
      table 2 since it adds nothing and averaging results across tasks should be
      avoided.


Learning with Noisy Labels
In this paper, we theoretically study the problem of binary classification in the presence of random classification noise -the learner, instead of seeing the true labels, sees labels that have independently been flipped with some small probability. Moreover, random label noise is class-conditional -the flip probability depends on the class. We provide two approaches to suitably modify any given surrogate loss function. First, we provide a simple unbiased estimator of any loss, and obtain performance bounds for empirical risk minimization in the presence of iid data with noisy labels. If the loss function satisfies a simple symmetry condition, we show that the method leads to an efficient algorithm for empirical minimization. Second, by leveraging a reduction of risk minimization under noisy labels to classification with weighted 0-1 loss, we suggest the use of a simple weighted surrogate loss, for which we are able to obtain strong empirical risk bounds. This approach has a very remarkable consequence -methods used in practice such as biased SVM and weighted logistic regression are provably noise-tolerant. On a synthetic non-separable dataset, our methods achieve over 88% accuracy even when 40% of the labels are corrupted, and are competitive with respect to recently proposed methods for dealing with label noise in several benchmark datasets.

Overall, I found the paper quite well-written.
      I think the assertion in the abstract (and then again in the
      Experiments section), that you learn with 88% accuracy when 80% labels
      are flipped is incorrect. You are only flipping 40% of the labels (the
      probability of flip is convex combination of \rho_+ and \rho_- and not
      their sum). In any, case the goal is to get better accuracy as noise
      rate goes to half, not one (where the problem is trivially easy with
      just all labels flipped). I wonder if it is worthwhile to
      compare your methods with Kearns' SQ setting. Since minimizing a
      convex loss can be done by gradient methods (which have statistical
      analogues). And so you would get tolerance to random classification
      noise for free. (Kearns does not allow class-conditional noise, but I
      think that part can be handled easily.) I'm not sure what kind of
      excess risk bounds you would get by such SQ-simulation. Minor
      quibbles: -------------- 1. You use the term PU learning, without
      ever defining it. 2. You use "so-called" very often. Especially for
      zero-one loss. Why is it so-called? If you have objection to the name
      you should state it. --- Update: Regarding using the SQ model.
      Even if you are using surrogate loss, your optimization problem can be
      solved using an algorithm that only makes statistical queries rather than
      data points.


Analyzing Hogwild Parallel Gaussian Gibbs Sampling
Sampling inference methods are computationally difficult to scale for many models in part because global dependencies can reduce opportunities for parallel computation. Without strict conditional independence structure among variables, standard Gibbs sampling theory requires sample updates to be performed sequentially, even if dependence between most variables is not strong. Empirical work has shown that some models can be sampled effectively by going "Hogwild" and simply running Gibbs updates in parallel with only periodic global communication, but the successes and limitations of such a strategy are not well understood. As a step towards such an understanding, we study the Hogwild Gibbs sampling strategy in the context of Gaussian distributions. We develop a framework which provides convergence conditions and error bounds along with simple proofs and connections to methods in numerical linear algebra. In particular, we show that if the Gaussian precision matrix is generalized diagonally dominant, then any Hogwild Gibbs sampler, with any update schedule or allocation of variables to processors, yields a stable sampling process with the correct sample mean.

The authors present analysis of "hogwild" parallel
      Gibbs sampling, where Gibbs updates are run for blocks of variables
      separately on each processor without communicating each iteration, so that
      the sampler is no longer "correct" at least in the traditional sense. The
      analysis focuses on sampling Gaussian graphical models, with the main
      result being that under a condition on the precision matrix (being
      generalised diagonally dominant) hogwild Gaussian Gibbs sampling is
      stable, and gives the correct asympoptic mean. The story for variances is
      a little more complex: a finite error is introduced when there are cross
      processor interactions that are ignored, and these become more important
      if the mixing at each processor is faster, but in the limit of drawing
      exact samples at each processor the variance can be corrected
      analytically. Quality. The ideas and analysis are novel and
      rigorous as far as I can tell. Clarity. The paper is reasonably
      well written but could be better organised, especially for the typically
      ML reader who is probably not overly familar with the linear algebra terms
      or the various linear solvers introduced. Having a "background material"
      section first laying this groundwork might be helpful.
      Originality. I am not aware of other analysis of this type of
      sampling. Significance. Scaling sampling methods is of significant
      interest to the NIPS community, so analysis of methods that attempt to do
      this is valuable, especially if it points the way towards algorithmic
      improvements. This paper laid out some nice ideas, many of which seem
      intuitive: hogwild sampling is dependent on the cross processor
      interactions not being too strong, and faster local mixing might actually
      be detrimental in some ways. However, the connection of these ideas to
      potential algorithm improvements was only hypothetical and then only
      hinted at. The paper begs the question in several cases. The generalised
      diagonally dominant condition in practice means I should try to put highly
      interacted variables: how should I try to do this? How well does it work
      in practice? Proposition 5 gives an analytic correct for the variance when
      q=inf: what happens if I use this for q large? I think this would have
      been a stronger paper if these questions had been addressed: if not
      empirically, at least in the discussion. But I accept there is a space
      limitation. A relevant reference that also used "hogwild" sampling
      is Finale Doshi-Velez, Shakir Mohamed, David A. Knowles and Zoubin
      Ghahramani. Large scale nonparametric Bayesian inference: Data
      parallelisation in the Indian buffet process. NIPS 2009. Minor
      notes Equation 1: am I missing something or there should be a minus
      sign?! Next equation (line 141): hat over v


Compete to Compute
Local competition among neighboring neurons is common in biological neural networks (NNs). In this paper, we apply the concept to gradient-based, backprop-trained artificial multilayer NNs. NNs with competing linear units tend to outperform those with non-competing nonlinear units, and avoid catastrophic forgetting when training sets change over time.

This paper introduces a simple form of nonlinearity
      into neural net architectures - forming groups of (typically two) neurons
      and zeroing out all the neurons in a group except the one with the highest
      value. While a very simple idea, it seems to give good results on
      classification. The authors also give evidence that the network does not
      forget as much as networks with more standard nonlnearities, however there
      might be a problem with that experiment (see below). This paper
      introduces a simple form of nonlinearity into neural net architectures -
      forming groups of (typically two) neurons and zeroing out all the neurons
      in a group except the one with the highest value. While a very simple
      idea, it seems to give good results on classification. The authors also
      give evidence that the network does not forget as much as networks with
      more standard nonlnearities, however there might be a problem with that
      experiment (see below). 1) There are better results on permutation
      invariant mnist, see table 1. in http://arxiv.org/pdf/1302.4389v3.pdf
      .Some of them are just feedforward networks. Also I don't agree that
      droput in the input should be considered a data augmentation since it
      doesn't assume anything about the input structure. You should have tried
      that experiment too. 2) It is a good property for the network not
      to forget. However the experiments could have few issues. You wait until
      network reaches certain likelihood and then change the data/labels. Since
      the new nonlinearity peforms better on recognition, it doesn't have to
      work as hard to reach the likelihood and so it doesn't need to do so much
      training, and so the reason it doesn't forget as much can simply be that
      it didn't train as much to forget what it has learned. 3) You
      should have also done a similar and related experiment. You have obtained
      the perofmance when you train on all digits at the same time (till
      convergence). Now, train till convergence one digits 1,2,3,4,5, then add
      the remaining digits and train on all digits. This tests how much is the
      network stuck in the minimum it found when training in the first phase.
      Ideally the performance in the second experiment (training on 1-5 and then
      training on 1-10) is the same as training on 1-10 from the start. It is
      known that this is not the case for sigmoid networks. Quality:
      Good, but more experiments should be there and the forgeting expriment
      should be better. Clarity: Very good Originality: I haven't seen
      it before. On negative side it is just another nonlinearity, related to
      max pooling, on the positive side it is a simple idea the gives good
      result. Significance: Somewhat significant - another nonlinearity into
      neural networks toolbox.


Noise-Enhanced Associative Memories
Recent advances in associative memory design through structured pattern sets and graph-based inference algorithms allow reliable learning and recall of exponential numbers of patterns. Though these designs correct external errors in recall, they assume neurons compute noiselessly, in contrast to highly variable neurons in hippocampus and olfactory cortex. Here we consider associative memories with noisy internal computations and analytically characterize performance. As long as internal noise is less than a specified threshold, error probability in the recall phase can be made exceedingly small. More surprisingly, we show internal noise actually improves performance of the recall phase. Computational experiments lend additional support to our theoretical analysis. This work suggests a functional benefit to noisy neurons in biological neuronal networks.

SUMMARY In this article, the authors study a
      model for associative memory where the patterns lie in some
      low-dimensional manifold. The structure of the patterns is captured by the
      so-called "constraint-neurons". Those constraint-neurons together with the
      pattern neurons form the memory network. The authors propose a recall
      algorithm for this network and show that intrinsic noise actually improves
      recall performance. The paper is well written and as far as I can
      judge, it is technically sound. MAJOR COMMENTS 1.
      Biological plausibility. The paper starts with considerations on
      associative memory and the brain. However, the associative memory model
      seems very distant to any biological implementation. - For example,
      how would step 4 in Algorithm 2 be implemented in some biologically
      realistic network? This step assumes that some memory should be kept over
      the tmax iterations of Algo 1 (c.f. Step 3 in Algo 3). - How should we
      think of the integer-valued state of the neuron (are states spike vs
      non-spike? this binary code (S = 2) would be different from the code in
      the constraint neurons.) 2. As mentioned by the authors, the
      architecture is similar to the one of Restricted Boltzman machines (RBM),
      the pattern neurons being the visible neurons and the constraint neuron
      being the hidden neurons. Now, with RBM, neurons are also intrinsically
      noisy and the level of intrinsic noise is adapted (through synaptic
      strength) such that the marginal distribution over visible units is
      consistent with the data. So I would also expect that the performance
      would be poorer with RBM if we forced neurons to be deterministic.
      Therefore, it is not clear to what extend it is a breaking news that
      intrinsic noise is good. Furthermore, the authors state that Deep Belief
      Nets (and therefore RBM as well) are used for classification and not
      pattern completion, but this is not correct. RBM can be used for pattern
      completion. 3 Other approaches to associative memories have
      been proposed. For example in [R1], Lengyel et al. see recall as a
      Bayesian inference process which exploits knowledge about the structure of
      the stored patterns (prior) and the knowledge about the learning rule as
      well as knowledge about the corrupting process of the inputs (the
      likelihood). In this perspective, the posterior distribution is a
      deterministic function of the prior and the likelihood and does not
      require additional noise. So how is it possible that the "best" recall
      rule (in a Bayesian sense) does not need noise (except if one want to
      sample from the posterior distribution) and the recall rule presented here
      needs noise? [R1] Lengyel, M., Kwag, J., Paulsen, O., & Dayan,
      P. (2005). Matching storage and recall: hippocampal spike timing–dependent
      plasticity and phase response curves. Nature Neuroscience, 8(12),
      1677–1683.


A Latent Source Model for Nonparametric Time Series Classification
For classifying time series, a nearest-neighbor approach is widely used in practice with performance often competitive with or better than more elaborate methods such as neural networks, decision trees, and support vector machines. We develop theoretical justification for the effectiveness of nearest-neighbor-like classification of time series. Our guiding hypothesis is that in many applications, such as forecasting which topics will become trends on Twitter, there aren't actually that many prototypical time series to begin with, relative to the number of time series we have access to, e.g., topics become trends on Twitter only in a few distinct manners whereas we can collect massive amounts of Twitter data. To operationalize this hypothesis, we propose a latent source model for time series, which naturally leads to a "weighted majority voting" classification rule that can be approximated by a nearest-neighbor classifier. We establish nonasymptotic performance guarantees of both weighted majority voting and nearest-neighbor classification under our model accounting for how much of the time series we observe and the model complexity. Experimental results on synthetic data show weighted majority voting achieving the same misclassification rate as nearest-neighbor classification while observing less of the time series. We then use weighted majority to forecast which news topics on Twitter become trends, where we are able to detect such "trending topics" in advance of Twitter 79% of the time, with a mean early advantage of 1 hour and 26 minutes, a true positive rate of 95%, and a false positive rate of 4%.

The authors examine theoretically and empirically
      properties of the k=1-nearest neighbor classification (NN) and a trivial
      variant of the kernel density classification, called “weighted majority
      voting”, in a time-series binary classification problem. For theoretical
      analysis, a simplified generative model for time-series is introduced.
      Under this model, they provide non-asymptotic performance guarantees in
      terms of how large of a training dataset and how much of the time-series
      length to be classified. Although the theoretical analysis is
      technically sound, the empirical work will be incomplete. In the
      experiments, the weighted majority voting and the NN are just tested while
      data analysts would at least use the k-NN, rather than the NN, with a
      tuned ‘k’ by cross-validation or so. I think the authors should apply k-NN
      as a baseline. Otherwise, I feel that they evaluate the presented results
      only from advantageous viewpoint. Also there exist several methods [i, ii,
      iii] similar to the (proposed) weighted majority voting. The
      linkage between theoretical and empirical parts seems to be week. While
      the presented upper error-bound of the NN seems to be always lower than or
      equal to that of the weighted majority voting, the weighted majority
      voting outperformed the NN in the experiments. Some careful discussion
      about this will be needed for the final version. Minors: - It
      would be better to explain why the summations over time shifts need to be
      replaced by the minimums in Eq. (7). - To understand the claims such
      in lines 284~286, where the constant factor of (\theta + 1/\theta) in Eq.
      (10) can be replace by 1, Remark in the supplementary material is
      essential. The authors should introduce the content in the remark in the
      main paper. - It would be helpful for readers if the bounds of Eq (10)
      and (13) are plotted with respected to G with various settings of \gamma
      and \sigma. - There is no conclusion or summary section. [i]
      Sahibsingh A. Dudani: The distance-weighted k-nearest neighbor rule, IEEE
      Transactions on System, Man, and Cybernetics, 6 (1976), 325-327.
      [ii]Thanh N. Tran, Ron Wehrens, Lutgarde M. C. Buydens: KNN-kernel
      density-based clustering for high-dimensional multivariate data.
      Computational Statistics & Data Analysis 51(2): 513-525 (2006).
      [iii] Jianping Gou, Taisong Xiong, Yin Kuang: A Novel Weighted Voting
      for K-Nearest Neighbor Rule. JCP 6(5): 833-840 (2011)


On Poisson Graphical Models
Undirected graphical models, such as Gaussian graphical models, Ising, and multinomial/categorical graphical models, are widely used in a variety of applications for modeling distributions over a large number of variables. These standard instances, however, are ill-suited to modeling count data, which are increasingly ubiquitous in big-data settings such as genomic sequencing data, user-ratings data, spatial incidence data, climate studies, and site visits. Existing classes of Poisson graphical models, which arise as the joint distributions that correspond to Poisson distributed node-conditional distributions, have a major drawback: they can only model negative conditional dependencies for reasons of normalizability given its infinite domain. In this paper, our objective is to modify the Poisson graphical model distribution so that it can capture a rich dependence structure between count-valued variables. We begin by discussing two strategies for truncating the Poisson distribution and show that only one of these leads to a valid joint distribution. While this model can accommodate a wider range of conditional dependencies, some limitations still remain. To address this, we investigate two additional novel variants of the Poisson distribution and their corresponding joint graphical model distributions. Our three novel approaches provide classes of Poisson-like graphical models that can capture both positive and negative conditional dependencies between count-valued variables. One can learn the graph structure of our models via penalized neighborhood selection, and we demonstrate the performance of our methods by learning simulated networks as well as a network from microRNA-sequencing data.

The paper provides a construction for multivariate
      distributions over unbounded counts that obey the Markov structure of
      an undirected network. Building a multivariate distribution
      over unbounded counts is in general a hard problem, as studied at
      length by Besag and others for the past 30 years (including textbooks
      such as Arnold et al.'s "Conditional Specification of Statistical
      Models", 1999, which extends some of the observations of the authors
      to other distributions such as conditionally-specified exponential
      distributions). In the end, the proposal given by the authors
      succeeds in some relevant ways. The upsides are constructions that do
      allow for marginal distributions over counts and which lead to
      relatively simple estimation algorithms. The downsides are, QPGM has
      marginally thinner tails and SPGM does not have closed-form
      conditional distributions (which somehow defeats the point of building
      a conditionally specified model). As a matter of fact, I don't even
      know how SPGM can be called a Poisson distribution (for QPGM at least
      one can claim that ``only'' the base measure is being changed). That's
      OK, but it made me wonder what the main motivation for modeling counts
      is, since the Poisson itself is not a good distribution to fit
      empirical data anyway (don't get me wrong, the Poisson is a very
      useful as a building block to many models - components of stochastic
      processes and within latent variable models etc. - but could you plot
      your data for breast cancer and tell me whether it looks anything
      close to a Poisson?). It would be very useful to have a plot of the
      probability mass function of the SPGM too, which feels somewhat
      convoluted at first sight. I suppose you are considering R and R0 as
      constants (or otherwise these wouldn't be exponential families). How
      are they chosen? Which advice do you give to the practitioner?
      That being said: to construct a multivariate distribution over
      counts obeying the independence model of MRF is hard. I honestly
      appreciate the effort put in this paper and I think the results are of
      theoretical interest to NIPS. The only thing that rubs me in the wrong
      way is the somewhat overly light appreciation of the literature. For
      instance, it almost feels like the authors don't really know what a
      copula is. The authors seem not to understand [8] (or at least
      definitely presented it in the wrong way), for instance, where the
      whole point is to build multivariate distributions for arbitrary
      discrete data (count data, inclusive), and for which a battery of MCMC
      and approximate inference methods exist. It made me wonder whether the
      discussion of [11] truly makes any sense, since the whole point of
      that pioneering book is to show how to build discrete models with
      log-linear parameters in a way it doesn't grow exponentially with the
      number of variables (although fair enough I don't have a copy of it
      with me right now and I don't remember anymore what it says about
      Poisson distributions). But perhaps the worst omission is a complete
      neglect of the vast spatial statistics literature, where
      high-dimensional count data analysis has been done for a long time.
      It needs to be said that several of these approaches (including
      the sparse precision Gaussian copula model of [8]) don't really model
      MRF-style independence constraints in the observable space. So as I
      said the theoretical contribution of this paper is a valid one. But as
      a practitioner I'm not yet convinced why I should pay the price of
      sticking to this model space instead of just using the simpler
      structured Gaussian random field + Poisson measurement model, which
      has been the standard for a long time. Final comment: I'm not
      an expert in gene expression analysis at all, but I would be grateful
      to have a reference newer than [20] claiming that ``counts of
      sequencing reads ... are replacing microarrays''.


On the Sample Complexity of Subspace Learning
A large number of algorithms in machine learning, from principal component analysis (PCA), and its non-linear (kernel) extensions, to more recent spectral embedding and support estimation methods, rely on estimating a linear subspace from samples. In this paper we introduce a general formulation of this problem and derive novel learning error estimates. Our results rely on natural assumptions on the spectral properties of the covariance operator associated to the data distribution, and hold for a wide class of metrics between subspaces. As special cases, we discuss sharp error estimates for the reconstruction properties of PCA and spectral support estimation. Key to our analysis is an operator theoretic approach that has broad applicability to spectral learning methods.

The paper deals with the problem of linear subspace
      estimation from samples, which appears in many applications such as PCA,
      kernel PCA and support estimation. The main contribution of the paper is
      that it provides a general formulation for the problem of subspace
      estimation and provides learning rates assuming certain decay behaviour
      for the eigenvalues of the covariance operator associated with the data
      distribution. As special cases, learning rates are provided for PCA and
      support estimation improving upon the existing results. Clarity:
      While the paper is technically interesting, the motivation presented in
      Section 1 is not clear. The paper is more of a technical exercise and
      lacks intuitive explanation. A better approach would have been to
      introduce the PCA problem (as discussed in Section 4.1) in Section 2 or
      may be even elaborate it in Section 1 so that the reader is introduced to
      the operator theoretic interpretation of PCA. With this motivation, one
      can generalize this notion as it is done in Section 2. I think this will
      improve the readability of the paper. As a minor comment, there are few
      typos, e.g., usage of "it is", which need to be fixed. Originality
      and Significance: The paper is technically interesting and is
      mathematically correct. Other comments: 1. In the paragraph
      below Eq. 2, it is mentioned that d_{\alpha,p} is a metric for
      0\le\alpha\le 1/2 and 1\le p\le\infty. However, in Proposition D.1 says
      that it is also a metric for 1/2\le \alpha\le 1. Its not clear why this
      restriction to 0\le\alpha\le 1/2? I dont see that such a restriction is
      needed at least at this point. 2. In the proof of Proposition D.1,
      it is mentioned that S_\rho=Ran C. Isn't it the closure of Ran C by
      Proposition D.3? 3. As mentioned above, it will be better to
      motivate the work through PCA and use proposition D.4 to motivate Eq. 2.
      While one might wonder are there any interesting values of \alpha and p
      other than 1/2 and 2, then you can present the application of set
      estimation where \alpha and p take values other than these. 4. In
      theorem 3.2, corollary 3.3 and remark 3.4, it is not mentioned that \alpha
      > 1/(rp). Without this condition, the estimator will not be consistent.
      This means for a fixed \alpha, p > 1/(r\alpha) which means the results
      do not hold for all 0\le p\le \infty as mentioned in the statements.
      Similar is the case with $\alpha$ where for a fixed p, the results hold
      only for \alpha > 1/(rp), or it should be assumed that r >
      max(1,1/(p\alpha)) 5. Line 126: postponed to "Section 7"
      6. The discussion about the non-requirement of regularizer from a
      statistical view point is not clear. Can you please elaborate what is
      making the difference compared to [11], that the proposed formulation does
      not require a regularizer even from statistical point of view. It is clear
      that some regularizer is needed for numerical stability but the lack of it
      is surprising from statistical sense. 7. While the proof technique
      is interesting and non-trivial, it appears that one can just do the entire
      analysis with p=2, i.e., Hilbert-Schmidt norm. Since the Schatten p-norm
      is a p-norm of the sequence of singular values (which can be countable in
      number as the operators considered are compact), it easy to check that
      ||A||_q\le ||A||_p for any 0 < p < q\le infty (because of the
      inclusion of l^p space in l^q). This means the analysis can simply by
      carried out using p=2 for any while bouding ||A||_p for any p\ge 2. This
      means classical Bernstein's inequality in Hilbert spaces can be used
      without needing sophisticated tools like Theorem B.1. Similarly for the
      case with 1\le p\le 2, one can bound ||A||_p by ||A||_1 in deriving from
      (14) onwards where one can use Holder and convert the analysis into
      bounding HS norms of certain operators. While I do not find
      anything wrong the current analysis and it is indeed nice, I am just
      wondering whether one needs the approach used in the paper to get sharp
      rates or one can still get such rates using the above mentioned analysis
      using HS norms. If one really needs the approach used in the paper to get
      sharp rates, this has to be highlighted along with a possible reason
      showing where HS approach fails.