A rotation-equivariant convolutional neural network model of primary visual cortex
Classical models describe primary visual cortex (V1) as a filter bank of orientation-selective linear-nonlinear (LN) or energy models, but these models fail to predict neural responses to natural stimuli accurately. Recent work shows that convolutional neural networks (CNNs) can be trained to predict V1 activity more accurately, but it remains unclear which features are extracted by V1 neurons beyond orientation selectivity and phase invariance. Here we work towards systematically studying V1 computations by categorizing neurons into groups that perform similar computations. We present a framework for identifying common features independent of individual neurons* orientation selectivity by using a rotation-equivariant convolutional neural network, which automatically extracts every feature at multiple different orientations. We fit this rotation-equivariant CNN to responses of a population of 6000 neurons to natural images recorded in mouse primary visual cortex using two-photon imaging. We show that our rotation-equivariant network outperforms a regular CNN with the same number of feature maps and reveals a number of common features, which are shared by many V1 neurons and are pooled sparsely to predict neural activity. Our findings are a first step towards a powerful new tool to study the nonlinear functional organization of visual cortex.
 The paper analyses the data collected from 6005 neurons in a mouse brain. Visual stimuli are presented and the responses of the neurons recorded. In the next step, a rotational equivariant neural network architecture together with a sparse coding read-out layer is trained to predict the neuron responses from the stimuli. Results show a decent correlation between neuron responses and trained network. Moreover, the rotational equivariant architecture beats a standard CNN with similar number of feature maps. The analysis and discussion of the results is interesting. Overall, the methodological approach is good. I have trouble understanding the plot in Figure 4, it also does not print well and is barely readable on paper. I have a small problem Figure 6 where "optimal" response-maps are presented. From my understanding, many of those feature maps are not looking similar to feature maps that are usually considered. Given the limited data available and the non-perfect modeling of neurons, the computed optimal response-map might include features that are not present in the dataset. Therefore, it would be interesting to compare those results with the stimuli used to gather the data. E.g. for a subset of neurons, one could pick the stimulus that created the maximum response and compare that to what the stimulus with the maximum response of the trained neuron was. It might be useful to include the average correlation of the neurons belong to each of the 16 groups(if there are any meaningful differences), especially as the cut-off of "correlation 0.2 on the validation set" is rather low. Note: I am not an expert in the neural-computation literature, I am adapting the confidence rating accordingly.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
A rotation-equivariant convolutional neural network model of primary visual cortex
Classical models describe primary visual cortex (V1) as a filter bank of orientation-selective linear-nonlinear (LN) or energy models, but these models fail to predict neural responses to natural stimuli accurately. Recent work shows that convolutional neural networks (CNNs) can be trained to predict V1 activity more accurately, but it remains unclear which features are extracted by V1 neurons beyond orientation selectivity and phase invariance. Here we work towards systematically studying V1 computations by categorizing neurons into groups that perform similar computations. We present a framework for identifying common features independent of individual neurons* orientation selectivity by using a rotation-equivariant convolutional neural network, which automatically extracts every feature at multiple different orientations. We fit this rotation-equivariant CNN to responses of a population of 6000 neurons to natural images recorded in mouse primary visual cortex using two-photon imaging. We show that our rotation-equivariant network outperforms a regular CNN with the same number of feature maps and reveals a number of common features, which are shared by many V1 neurons and are pooled sparsely to predict neural activity. Our findings are a first step towards a powerful new tool to study the nonlinear functional organization of visual cortex.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 The paper analyses the data collected from 6005 neurons in a mouse brain. Visual stimuli are presented and the responses of the neurons recorded. In the next step, a rotational equivariant neural network architecture together with a sparse coding read-out layer is trained to predict the neuron responses from the stimuli. Results show a decent correlation between neuron responses and trained network. Moreover, the rotational equivariant architecture beats a standard CNN with similar number of feature maps. The analysis and discussion of the results is interesting. Overall, the methodological approach is good. I have trouble understanding the plot in Figure 4, it also does not print well and is barely readable on paper. I have a small problem Figure 6 where "optimal" response-maps are presented. From my understanding, many of those feature maps are not looking similar to feature maps that are usually considered. Given the limited data available and the non-perfect modeling of neurons, the computed optimal response-map might include features that are not present in the dataset. Therefore, it would be interesting to compare those results with the stimuli used to gather the data. E.g. for a subset of neurons, one could pick the stimulus that created the maximum response and compare that to what the stimulus with the maximum response of the trained neuron was. It might be useful to include the average correlation of the neurons belong to each of the 16 groups(if there are any meaningful differences), especially as the cut-off of "correlation 0.2 on the validation set" is rather low. Note: I am not an expert in the neural-computation literature, I am adapting the confidence rating accordingly.
assistant
 The paper analyses the data collected from 6005 neurons in a mouse brain. Visual stimuli are presented and the responses of the neurons recorded. In the next step, a rotational equivariant neural network architecture together with a sparse coding read-out layer is trained to predict the neuron responses from the stimuli. Results show a decent correlation between neuron responses and trained network. Moreover, the rotational equivariant architecture beats a standard CNN with similar number of feature maps. The analysis and discussion of the results is interesting. Overall, the methodological approach is good. I have trouble understanding the plot in Figure 4, it also does not print well and is barely readable on paper. I have a small problem Figure 6 where "optimal" response-maps are presented. From my understanding, many of those feature maps are not looking similar to feature maps that are usually considered. Given the limited data available and the non-perfect modeling of neurons, the computed optimal response-map might include features that are not present in the dataset. Therefore, it would be interesting to compare those results with the stimuli used to gather the data. E.g. for a subset of neurons, one could pick the stimulus that created the maximum response and compare that to what the stimulus with the maximum response of the trained neuron was. It might be useful to include the average correlation of the neurons belong to each of the 16 groups(if there are any meaningful differences), especially as the cut-off of "correlation 0.2 on the validation set" is rather low. Note: I am not an expert in the neural-computation literature, I am adapting the confidence rating accordingly.