Published as a conference paper at ICLR 2018 SPHERICAL CNNS
Convolutional Neural Networks (CNNs) have become the method of choice for learning problems involving 2D planar images. However, a number of problems of recent interest have created a demand for models that can analyze spherical images. Examples include omnidirectional vision for drones, robots, and autonomous cars, molecular regression problems, and global weather and climate modelling. A naive application of convolutional networks to a planar projection of the spherical signal is destined to fail, because the space-varying distortions introduced by such a projection will make translational weight sharing ineffective. In this paper we introduce the building blocks for constructing spherical CNNs. We propose a definition for the spherical cross-correlation that is both expressive and rotation-equivariant. The spherical correlation satisfies a generalized Fourier theorem, which allows us to compute it efficiently using a generalized (non-commutative) Fast Fourier Transform (FFT) algorithm. We demonstrate the computational efficiency, numerical accuracy, and effectiveness of spherical CNNs applied to 3D model recognition and atomization energy regression.
Summary:
The paper proposes a framework for constructing spherical convolutional networks (ConvNets) based on a novel synthesis of several existing concepts.  The goal is to detect patterns in spherical signals irrespective of how they are rotated on the sphere.  The key is to make the convolutional architecture rotation equivariant.
Pros:
+ novel/original proposal justified both theoretically and empirically
+ well written, easy to follow
+ limited evaluation on a classification and regression task is suggestive of the proposed approach's potential
+ efficient implementation
Cons:
- related work, in particular the first paragraph, should compare and contrast with the closest extant work rather than merely list them
- evaluation is limited; granted this is the nature of the target domain
Presentation:
While the paper is generally written well, the paper appears to conflate the definition of the convolutional and correlation operators?  This point should be clarified in a revised manuscript.  
In Section 5 (Experiments), there are several references to S^2CNN.  This naming of the proposed approach should be made clear earlier in the manuscript.  As an aside, this appears a little confusing since convolution is performed first on S^2 and then SO(3). 
Evaluation:
What are the timings of the forward/backward pass and space considerations for the Spherical ConvNets presented in the evaluation section?  Please provide specific numbers for the various tasks presented.
How many layers (parameters) are used in the baselines in Table 2?  If indeed there are much less parameters used in the proposed approach, this would strengthen the argument for the approach.  On the other hand, was there an attempt to add additional layers to the proposed approach for the shape recognition experiment in Sec. 5.3 to improve performance?
Minor Points:
- some references are missing their source, e.g., Maslen 1998 and Kostolec, Rockmore, 2007, and Ravanbakhsh, et al. 2016.
- some sources for the references are presented inconsistency, e.g., Cohen and Welling, 2017 and Dieleman, et al. 2017
- some references include the first name of the authors, others use the initial 
- in references to et al. or not, appears inconsistent
- Eqns 4, 5, 6, and 8 require punctuation
- Section 4 line 2, period missing before "Since the FFT"
- "coulomb matrix" --> "Coulomb matrix"
- Figure 5, caption: "The red dot correcpond to" --> "The red dot corresponds to"
Final remarks:
Based on the novelty of the approach, and the sufficient evaluation, I recommend the paper be accepted.