Optimal prior-dependent neural population codes under shared input noise
The brain uses population codes to form distributed, noise-tolerant representations of sensory and motor variables. Recent work has examined the theoretical optimality of such codes in order to gain insight into the principles governing population codes found in the brain. However, the majority of the population coding literature considers either conditionally independent neurons or neurons with noise governed by a stimulus-independent covariance matrix. Here we analyze population coding under a simple alternative model in which latent "input noise" corrupts the stimulus before it is encoded by the population. This provides a convenient and tractable description for irreducible uncertainty that cannot be overcome by adding neurons, and induces stimulus-dependent correlations that mimic certain aspects of the correlations observed in real populations. We examine prior-dependent, Bayesian optimal coding in such populations using exact analyses of cases in which the posterior is approximately Gaussian. These analyses extend previous results on independent Poisson population codes and yield an analytic expression for squared loss and a tight upper bound for mutual information. We show that, for homogeneous populations that tile the input domain, optimal tuning curve width depends on the prior, the loss function, the resource constraint, and the amount of input noise. This framework provides a practical testbed for examining issues of optimality, noise, correlation, and coding fidelity in realistic neural populations.
Sensory systems in the brain are faced with the challenge of representing noisy signals with finite resources. Since these questions were first formalised and discussed over 50 years ago, many have pursued the optimal way of allocating resources to this task, and asked whether the brain's allocation of resources reflects this challenge. This paper attempts to further the body of literature devoted to this question. In particular, it attempts three goals: (1) to replace previous assessments of optimal coding that were based on numerical simulations with analytical results; (2) to study the implications of several components of these models (stimulus priors, the choice of objective function, the integration time, and the input noise) for optimal coding; and (3) to make predictions about how real neurons should behave under the kinds of noise considered here.Quality: Under the (mostly reasonable) assumptions made by the authors, I believe they make some headway on the first goal, namely to analytically derive some of the previous numerical results on optimal coding. There is no discussion on how robust these results are to deviations from these assumptions. The parameter dependence is a bit of a mixed bag -- while the results on adding noise lead to generally intuitive conclusions (e.g. under noisy conditions you can't make the tuning widths arbitrarily small), the time window and objective function results I believe are already mostly known (to some extent covered across Bethge et al 2002, Brunel & Nadal 1998, Wang et al 2012, and others perhaps, eg Berens 2011 for neurometrics). The part that worries me the most though is that the third goal of this paper (presented in Section 4), seems completely tangential to the remaining work, is poorly explained, and lead to predictions that are simply not borne out by the data. The authors are able to cherry-pick two data examples which only roughly follow the predicted patterns; even in this case the predictions are a complete order of magnitude out of proportion. While the authors admit this is a weak correspondence (Line 357: "[the results] are more an exception than a rule"), the authors don't evaluate the implications of this objectively (rather, they cite an unpublished conference talk [16], as a rather weak defence). The paper would be better if Section 4 were omitted, and the extra space devoted to better explanations of the derivations and intuitions.Some other comments:- The tiling property in eqn (3) is *only* an approximation, and does not hold exactly (as asserted) with finite numbers of neurons. What is the implication of the discrepancy? Does it make the decoder biased? Also this means that "the relevant stimulus range" should be defined as being well away from $\star{s}_1$ and $\star{s}_N$- Equation (11). A reference is necessary for this equation: it took me some digging. Also some comment is necessary about the fact that this is a complex-argument extension of the incomplete gamma function, since the second argument is negative, and the $-\lambda$ factor is raised to a fractional power (which is thus complex-valued). Things always have the potential to get very messy in the complex domain (which root of $-\lambda$? do we cross a branch point?), and I can't verify that something funky hasn't happened here.- No information was provided on how to obtain the bound in equation (12). I can't verify this.- Also, $\sigma_{tc}$ as a symbol is a stumbling block, and would be much cleaner as $\sigma_t$.There were several typographical errors which can be corrected.- eqn (4): $\exp^{-\lambda}$, not $\exp^{-R}$- Line 180: missing a factor of $2 \pi e$ inside the logarithm.- eqn (11): exponent on $-\lambda$ should be negative. Same for equation 22.- Fig 5 caption: s* = -5? not -2? Clarity: While the abstract and introduction are clear, I found that the mathematics were often poorly explained. In particular, many of the results were reiterating previous findings of others (e.g. Bethge et al 2002, Berens et al 2011) that had been obtained through other analyses, but these weren't cited throughout. For example, Fig 2B seems to parallel Berens 2011 Figs 2A-B (though why is the FI different as $\sigma_{tc}$ goes to zero?). The comments on pp 185-186 are well known (and indicated as such in the intro), and this should again be stated here. I really could not follow the logic in Section 4 particularly well, not for want of trying; it didn't help that crucial information on Lines 370-372 appeared in the discussion rather than here. Also, very importantly -- there really needs to be consolidation at the end of every section/subsection otherwise this appears as an enumeration of random results.- Line 121: what is S? The whole example is very confusing and unexplained.- Line 124: remind that the rate is independent of s.- Line 132: This is very unclear. Rather: "Since $\Prod_i \exp^{-f_i(s)} = \exp^{-\lambda}$ is constant, the likelihood on $s$ in (1) is thus proportional..."- Line 134: the normal would be clearer as $\mathcal{N}( s | 1/R ... )$- Line 146: The Fisher information, given by $FI(s) = E[ ... ] $- Fig 5: the top line and bottom line need to be distinguished within the figure itself. The interpretations of the left and middle panels of the top line are impossible to see.- Cramer -> CramérOriginality: This paper does extend the previous work on this question with a new treatment. As mentioned above, it often wasn't clear what was a new contribution and what was a re-derivation or re-presentation. For those results which had previously been numerically derived, there is little comment on whether these lead to the same conclusions as the previous results obtained analytically (especially since the authors emphasise that this is a key contribution of the paper).Significance: While the analytic treatment provides an extra degree of rigour above the previous numerical results, I am not convinced that this contribution alone is substantial. The extension to the input noise provide some results that may of use to some. However, the overall value of this paper is tainted by the lack of clear explanations or useful intuitions going along. Previous published work on this material is certainly clearer. Given that these optimality arguments have generally failed to make accurate predictions about coding in the brain in the past (with a few exceptions), and the predictions here are rather shoddy, this work convinces me more that this is a failed project than one worth pursuing. I doubt that is the authors' intention.