A Bayesian method for reducing bias in neural representational similarity analysis
In neuroscience, the similarity matrix of neural activity patterns in response to different sensory stimuli or under different cognitive states reflects the structure of neural representational space. Existing methods derive point estimations of neural activity patterns from noisy neural imaging data, and the similarity is calculated from these point estimations. We show that this approach translates structured noise from estimated patterns into spurious bias structure in the resulting similarity matrix, which is especially severe when signal-to-noise ratio is low and experimental conditions cannot be fully randomized in a cognitive task. We propose an alternative Bayesian framework for computing representational similarity in which we treat the covariance structure of neural activity patterns as a hyperparameter in a generative model of the neural data, and directly estimate this covariance structure from imaging data while marginalizing over the unknown activity patterns. Converting the estimated covariance structure into a correlation matrix offers a much less biased estimate of neural representational similarity. Our method can also simultaneously estimate a signal-to-noise map that informs where the learned representational structure is supported more strongly, and the learned covariance matrix can be used as a structured prior to constrain Bayesian estimation of neural activity patterns. Our code is freely available in Brain Imaging Analysis Kit (Brainiak) (https://github.com/IntelPNI/brainiak).
The manuscrit presents a technical bias that has impaired accurate estimation of the structure of similiarity between neural representations, i.e. the similarity between neural representations of distinct objects or percepts, from neuroimaging data (especially fMRI data). The manuscript first details mathematically how the bias emerges in the covariance matrix, especially when signal-to-noise ratio is low and the timing of presentations of the different objects are not well balanced. Then authors propose a new method for structure inference based on maximum likelihood estimation from a generative model of neural data with various parameters (covariance structure, SNR in all voxels, variance and autoregressive coefficient of noise). This method naturally gets rid of the estimation bias, as is shown with synthetic data. It is finally applied to a fMRI dataset showing similarity between a behaviourally derived and a neurally derived similarity structure of representations of animals in human subjects.
The manuscript is very clearly written, the method is impressive from the technical point of view and it seems to address very appropriately a real issue for analysis of neuroimaging data. I expect that it can be of great use to many neuroscientists by improving the estimation of similarity structure from neural activity. Moreover the perspective of having improved measure of betas in fMRI (notably taking into account the bias in covariance matrix and the smooth prior for SNR) is extremely promising.   This being said, here is a series of additional comments: -The authors argue that alternate methods are not well grounded, but it would be nice to have the illustration in Figure 2 that the new method actually performs better at recovering the covariance structure in synthetic data. - I have a  concern regarding the spatial independence of noise. If noise is not spatially independent (and it generally isn't, as acknowledged in the discussion), this will create some consistent overestimation the correlational structure. Will this affect more the estimation than the original bias that the method seeks to get rid off ? In other terms, is the cure worse than the disease ? - It would be great if the code could be made public, and perhaps interfaced with one of the popular fMRI analyses softwares/toolboxes such as SPM.  - maybe in figure 3 correlation between behaviourally and neural obtained matrices could be used as a measure of the correspondance between the two.  *Writing/typos: - L35-36 “ what stimuli have very different representations” → rephrase - L41 “demonstrations” -L42 “highly similar representational structure” unclear - L59-60 maybe specify that when Y is white noise, there is still structure in design matrix X (which is where the 'representational' structure comes from). It cost me quite a lot to understand that point. - L94 “is”->”are” L214 “advantage not available” → either “advantage over” or “feature not available” L230 “intensities” L250 “experiment design”  [EDIT AFTER REBUTTAL] all my concerns were convincingly addressed, except the one about testing the bayesian RSA against alternative methods described l268-287 on synthetic data.