Fisher-Optimal Neural Population Codes for High-Dimensional Diffeomorphic Stimulus Representations
In many neural systems, information about stimulus variables is often represented in a distributed manner by means of a population code. It is generally assumed that the responses of the neural population are tuned to the stimulus statistics, and most prior work has investigated the optimal tuning characteristics of one or a small number of stimulus variables. In this work, we investigate the optimal tuning for diffeomorphic representations of high-dimensional stimuli. We analytically derive the solution that minimizes the L 2 reconstruction loss. We compared our solution with other well-known criteria such as maximal mutual information. Our solution suggests that the optimal weights do not necessarily decorrelate the inputs, and the optimal nonlinearity differs from the conventional equalization solution. Results illustrating these optimal representations are shown for some input distributions that may be relevant for understanding the coding of perceptual pathways.
Summary Finding the objective functions that 
      regions of the nervous system are optimized for is a central question in 
      neuroscience, providing a central computational principle behind neural 
      representation in a given region. One common objective is to maximize the 
      Shannon Information the neural response encodes about the input (infomax). 
      This is supported by some experimental. Another is to minimize the 
      decoding error when the neural population is decoded for a particular 
      variable or variable. This has also been found to have some experimental 
      evidence. These two different objectives are similar in some 
      circumstances, giving similar predictions, in other cases they differ 
      more. Studies finding model optimal distributions of neural 
      population tuning that minimizes decoding error (L2-min) have mostly 
      considered 1-dimensional stimuli. In this paper the authors extend 
      substantially on this, by developing analytical methods for finding the 
      optimal distributions of neural tuning for higher dimensional stimuli. 
      Their methods apply under certain limited conditions , such as when there 
      is an equal number of neurons as stimulus dimensions (diffeomorphic). The 
      authors compare their results to the infomax solution (in most detail for 
      the 2D case), and find fairly similar results in some respects, but with 
      two key differences. That the L2-min basis functions are more orthogonal 
      than the infomax, and that the L2-min has discrete solutions rather than 
      the continuum found for infomax. A consequence of these differences is 
      that L2-min representations encode more correlated signals. On 
      the quality score, impact score and confidence score. I chose accept 
      (top 50%) as I thought it was a good and interesting paper. I chose 2 
      (although this is a rather course grain measure, I probably would go for 
      1.7). I chose 3 (fairly confident) but if I could chose 2.5 I would. I 
      think I got the gist of the paper, and Sections 1-2 were clear. I got lost 
      in several place in Sections 3-6 but I think I followed the general 
      argument to a varying degree. Section 7 was clear. Quality 
      Is the paper technically sound? As far as I can tell, the 
      paper is technically sound. The areas of the paper that I could 
      confidently follow held up in my opinion. Are the claims well 
      supported? I would say that the claims are well supported as well as I 
      could follow them. A little more explanation as to why the more orthogonal 
      solution of the L2-min encodes correlated signals would be useful. 
      Is this a complete piece of work? For a NIPS paper I would say 
      yes. Are the authors careful (and honest) about evaluating both 
      the strengths and weaknesses of the paper? Yes, the authors are very 
      careful and honest in evaluating the paper’s strength and weaknesses. In 
      fact, I think they are a little too circumspect, and I would have liked 
      just a little more discussion of its strengths and possibilities. However, 
      I recognize there are space limits. Clarity Is the 
      paper clearly written? Where I understood the paper is seems 
      moderately clearly written. Where I didn’t understand I don’t know if it 
      was the authors lack of clarity or my lack of familiarity with what they 
      were doing. The clarity and consistency of the notation could be 
      improved. The notation varies somewhat across the paper and terms are 
      sometimes not defined. For example the don’t define E[ |s] and <> as 
      expectation. They use n for the number of neurons and the number of 
      spikes. They use ‘ for differentiation but also as simply a marker (as s’ 
      in 2.4). Is the paper well organized? Yes. Does the 
      paper adequately inform the reader? I would have liked some areas 
      spelt out more. See above. Originality Are the 
      problems or approaches new? Yes, analytical methods for high 
      dimensional L2-min representation are new as far as I know. Is it 
      clear how this work differs from previous contributions? Yes. 
      Is related work adequately referenced? Yes. 
      Significance Are these results important? This 
      work could help form the basis of a more general research program that 
      examines what general objective functions particular neural systems use. 
      Thus I think the results are important. Are other people likely to 
      use these ideas? Yes, I think they could be useful as the basis of 
      future more specific and empirical analyses, and also expanded into a more 
      general framework. Does the paper address a difficult problem in a 
      better way than previous research? Yes, the analytical results are 
      more flexible and secure and can be used to speed up numerical methods. 
      Does it advance the state of the art? Yes Does it 
      provide a unique theoretical approach? Yes. Strengths 
      High-dimensional L2-min is novel. Novel analytical approach. 
      Comparison with Infomax. Clear differences in solutions given. 
      Weaknesses Restricted circumstances (diffeomorphic). A 
      little unclear in places (may be due to reviewer). Detailed 
      review Sections 1 and 2 are clear to me. Only minor points 
      070: As opposed to the encoding, the 070: function called 
      080: E[|s] is the expectation over r, but might be good to define 
      it. 098: Perhaps you are using a notation I am not familiar with, 
      but by analogy with the L2 norm, should it be the L22 loss? 101: 
      In this equation I am more used to a ≥. Please define the meaning of the 
      curved ≥ in this context. 113: Define s’. Later ‘ is used for 
      derivative, please clear up this ambiguity. In section 3 I get 
      somewhat confused. Please give more detail on how Holder’s inequality is 
      used to find the minimum? 163: …, the right side of Equation 10 
      is… 178: Use n for number of spikes in the appendix, need to 
      change one of these variables. Section 4. Also found this somewhat 
      confusing. 189: double integral sign for a multi-dimensional 
      integral is confusing. I suggest using placing … between them, or just 
      having one integral sign. 193: as 189. Sections 5 and 6. 
      Found this somewhat confusing, but less than 3 and 4. 229: What 
      happens to |wk| in the relationship between Equations 17 and 18? Confused 
      here. 241: “Minimizing the numerator Var[wkTs] makes…” The 
      numerator of which equation? I assume you are referring to equation 17, if 
      so the relationship between Var[wkTs] and the numerator needs to be made 
      more explicit. 321: Might want to briefly discuss the similarity 
      and differences between Equation 16 and Equation 28. 371: 
      Apologies if I am totally missing the point. I would imagine that a more 
      orthogonal basis would results in less correlated signals. Where am I 
      wrong? Please explain. A little more elaboration on this point in the text 
      would be useful. Section 7. Generally clear. Section A. 
      Clear.