ROTDCF: DECOMPOSITION OF CONVOLUTIONAL FIL- TERS FOR ROTATION-EQUIVARIANT DEEP NETWORKS
Explicit encoding of group actions in deep features makes it possible for convolutional neural networks (CNNs) to handle global deformations of images, which is critical to success in many vision tasks. This paper proposes to decompose the convolutional filters over joint steerable bases across the space and the group geometry simultaneously, namely a rotation-equivariant CNN with decomposed convolutional filters (RotDCF). This decomposition facilitates computing the joint convolution, which is proved to be necessary for the group equivariance. It significantly reduces the model size and computational complexity while preserving performance, and truncation of the bases expansion serves implicitly to regularize the filters. On datasets involving in-plane and out-of-plane object rotations, Rot-DCF deep features demonstrate greater robustness and interpretability than regular CNNs. The stability of the equivariant representation to input variations is also proved theoretically. The RotDCF framework can be extended to groups other than rotations, providing a general approach which achieves both group equivariance and representation stability at a reduced model size.
This work extends on [1] by constructing CNN filters using Fourier-Bessel (FB) bases for rotation equivariant networks. Additionally to [1] it extends the process with using SO(2) bases which allow to learn combination of rotated FB bases and ultimately achieve good performance with less parameters than standard CNN networks thanks to filter truncation.
In general, this work is well written and shows interesting results. However it lacks context with regards to other existing works. For example [2] also uses steerable filters for achieving rotation equivariance, however with different steerable bases (rotation harmonics instead of FB). It would be useful to clarify why FB bases are more appropriate for truncation, eventually providing empirical evidence (even though rotation harmonics would probably need more parameters). Authors mention [2], however disregard it due to computational complexity, which would be the same if the rotation harmonics bases were truncated as well.
Similarly, this work is not strong in evaluating against existing methods. It provides evaluation of the vanilla group equivariant networks in a similar configuration, but due to design choices in the training and test set, it is not possible to compare it against other algorithms and other steerable bases such as those from [2]. This degrades the results slightly as it does not allow to verify the baseline results from other works.
Additionally, it would be useful to provide an ablation study which would show how important the bases in SO(2) are important for the model accuracy. This would allow to compare the results against the [1] as the FB filters are steerable as well (Equation 4).
It is hard to reach a final rating for this submission. On one hand, it can be seen as an incremental improvement of [1] for a new domain of tasks, without a thorough comparison against existing methods. On the other hand, the paper is well written and the results look promising - evaluation verifies that the algorithm performs well in multiple tasks with a fraction of parameters.
Considering that authors plan to release the source code and that this conference aims for publishing novel ideas (and the goal of this work is to achieve rotation equivariance with less parameters, which hasn't been tackled before), I am inclined towards acceptance of this paper, even though the experiments can be significantly improved.
Unfortunately, I was not able to verify correctness of the provided proofs.
Additional minor issues:
* The paper does not specify what FB bases exactly are being used (such as in [table 1;1]), mainly it does not seem to specify the SO(2) bases.
* It would be useful to visualise K and K_\alpha in Figure 1.
* Citations, if not part of the sentence, should be in parentheses to improve readability (\citep for natbib).
* On page 8, end of first paragraph - wrong reference (see S.M.)
* L, in section 2.3 is not defined.
[1] Qiu, Qiang, et al. "DCFNet: Deep Neural Network with Decomposed Convolutional Filters.", ICML 2018
[2] Weiler, Maurice, et al. “Learning Steerable Filters for Rotation Equivariant CNNs.” CVPR 2018