A ROTATION-EQUIVARIANT CONVOLUTIONAL NEURAL NETWORK MODEL OF PRIMARY VISUAL CORTEX
Classical models describe primary visual cortex (V1) as a filter bank of orientationselective linear-nonlinear (LN) or energy models, but these models fail to predict neural responses to natural stimuli accurately. Recent work shows that convolutional neural networks (CNNs) can be trained to predict V1 activity more accurately, but it remains unclear which features are extracted by V1 neurons beyond orientation selectivity and phase invariance. Here we work towards systematically studying V1 computations by categorizing neurons into groups that perform similar computations. We present a framework for identifying common features independent of individual neurons' orientation selectivity by using a rotation-equivariant convolutional neural network, which automatically extracts every feature at multiple different orientations. We fit this rotation-equivariant CNN to responses of a population of 6000 neurons to natural images recorded in mouse primary visual cortex using two-photon imaging. We show that our rotation-equivariant network outperforms a regular CNN with the same number of feature maps and reveals a number of common features, which are shared by many V1 neurons and are pooled sparsely to predict neural activity. Our findings are a first step towards a powerful new tool to study the nonlinear functional organization of visual cortex.
In this interesting study, the authors show that incorporating rotation-equivariant filters  (i.e. enforcing weight sharing across filters with different orientations) in a CNN model of the visual system is a useful prior to predict responses in V1. After fitting this model to data, they find that the RFs of model V1 cells do not resemble the simple Gabor filters of textbooks, and they present other quantitative results about V1 receptive fields. The article is clearly written and the claims are supported by their analyses. It is the first time to my knowledge that a rotation-equivariant CNN is used to model V1 cells.
The article would benefit from the following clarifications:
1. The first paragraph of the introduction discusses functional cell types in V1, but the article does not seem to reach any new conclusion about the existence of well-defined clusters of functional cell types in V1. If this last statement is correct, I believe it is misleading to begin the article with considerations about functional cell types in V1. Please clarify.
2. For clarity, it would help the reader to mention in the abstract, introduction and/or methods that the CNN is trained on reproducing V1 neuron activations, not on an image classification task as in many other studies (Yamins 2014, etc). 
3. “As a first step, we simply assume that each of the 16 features corresponds to one functional cell type and classify all neurons into one of these types based on their strongest feature weight.” and “The resulting preferred stimuli of each functional type are shown in Fig. 6.“
Again, I think these statements are misleading because they suggest that V1 cells indeed cluster in distinct functional cell types rather than form a continuum. However, from the data shown, it is unclear whether the V1 cells recorded form a continuum or distinct clusters. Unless this question is clarified and the authors show the existence of functionally distinct clusters in their data, they should preferably not mention "cell types" in the text.
Suggestions for improvement and questions (may not necessarily be addressed in this paper):
4. “we apply batch normalization”
What is the importance of batch normalization for successfully training the model? Do you think that a sort of batch normalization is implemented by the visual system? 
5. “The second interesting aspect is that many of the resulting preferred stimuli do not look typical standard textbook V1 neurons which are Gabor filters. ”
OK but the analysis consists of iteratively ascending the gradient of activation of the neuron from an initial image. This cannot be compared directly to the linear approximation of the V1 filter that is computed experimentally from doing a spike-triggered average (STA) from white noise. A better comparison would be to do a single-step gradient ascent from a blank image. In this case, do the filters look like Gabors?
6. Did you find any evidence that individual V1 neurons are themselves invariant to a rotation?
7. The article could be more self-contained. There are a lot of references to Klindt et al. (2017) on which this work is based, but it would be nice to make the article understandable without having to read this other article.
Typo: Number of fearture maps in last layer 
Conclusion:
I believe this work is significant and of interest for the rest of the community studying the visual system with deep networks, in particular because it finds an interesting prior for modeling V1 neurons, that can probably be extended to the rest of the visual system. However, it would benefit from the clarifications mentioned above.