Demixing odors -fast inference in olfaction
The olfactory system faces a difficult inference problem: it has to determine what odors are present based on the distributed activation of its receptor neurons. Here we derive neural implementations of two approximate inference algorithms that could be used by the brain. One is a variational algorithm (which builds on the work of Beck. et al., 2012), the other is based on sampling. Importantly, we use a more realistic prior distribution over odors than has been used in the past: we use a "spike and slab" prior, for which most odors have zero concentration. After mapping the two algorithms onto neural dynamics, we find that both can infer correct odors in less than 100 ms. Thus, at the behavioral level, the two algorithms make very similar predictions. However, they make different assumptions about connectivity and neural computations, and make different predictions about neural activity. Thus, they should be distinguishable experimentally. If so, that would provide insight into the mechanisms employed by the olfactory system, and, because the two algorithms use very different coding strategies, that would also provide insight into how networks represent probabilities.
Comments: This paper describes two candidate 
      mechanisms for implementing probabilistic inference in the olfactory 
      system, one based on variational inference and a second based on 
      sampling. The basic problem is to infer the (marginal probability 
      over) the identity and concentration of odors present in a sparse 
      mixture from the noisy responses of a population of olfactory receptor 
      neurons. The ORNs respond linearly with Poisson noise, and the 
      proposed algorithms perform inference under a spike-and-slab prior on 
      odors. The problem is clearly motivated and the work itself is 
      very timely given the recent surge of interest in mechanisms for 
      performing Bayesian inference in neural circuits (and in the olfactory 
      system in particular). The ideas are novel and the performance seems 
      impressive; I'm surprised by the improvement over template matching. 
      This is certainly one of the more creative and more interesting papers 
      I reviewed this year. The paper seems well suited to NIPS 
      insofar as it brings together theoretical ideas about algorithms for 
      probabilistic inference and practical ideas about how to implement 
      them in neural hardware (a consideration that does not arise in 
      standard machine learning papers). Comments: - 
      Variational algorithm: the intuition for the steps in deriving the 
      variational approximation (pg 3) are not exceedingly clear, and some 
      of the assumptions seem like they might be rather severe. In light of 
      this, it seems surprising that the variational algorithm works so 
      well, that is, that the sampling-based estimate isn't more accurate. 
      - Sampling algorithm: some clever ideas here. I like the trick for 
      changing variables to \tilde c_j and the idea for asynchronous Gibbs 
      sampling. However, the relationship between s' and \tilde c (the 
      variables being sampled) and neural activity in the network should 
      be spelled out more clearly. - Concern about plausibility: the 
      inference algorithms both work in real time, but the ORN responses r_i 
      seem to be fixed (discrete) spike counts, which doesn't seem well 
      matched. Can the same algorithms be applied with point process ORN 
      responses? Also, it would help to include some detail about how 
      realistic the assumptions about connectivity and number of ORNs is 
      (for simulations described in Section 4). - It would be nice 
      to have some more explicit predictions about the form of the neural 
      activity itself (i.e., as opposed to predictions about the 
      relationship between neural activity and probability). The prediction 
      of "increased variability" for sampling-based inference doesn't seem 
      very specific, since one could also implement a dynamical system for 
      variational inference in which the individual responses are very 
      noisy. Are there any data currently in existence that might be 
      compared to model predictions? Minor comments: - eq 2.2a: 
      subscripts on alpha and Beta seem slightly puzzling, especially since 
      they're missing in eq. 2.3. (It's not till we see the variational 
      approximation section that we see why they need to be distinguished 
      from alpha_0 and Beta_0). - eq 2.2b: should say what \pi is 
      - pg 2, line 098: not clear what it means that alpha_0 and and 
      Beta_0 can model the effects of the baseline firing rate. - pg 
      2: clever use of multinomial theorem, but the N_ij need clearer 
      definition. The "N_ij is shorthand for..." sentence is very 
      confusing. Indices of summation (j=1 to to r_j?) should be given in eq 
      3.2. - I wonder if there's any particular reason to use 
      Langevin sampling (as opposed to either vanilla Metropolis-Hastings or 
      something fancier).