PerforatedCNNs: Acceleration through Elimination of Redundant Convolutions
We propose a novel approach to reduce the computational cost of evaluation of convolutional neural networks, a factor that has hindered their deployment in lowpower devices such as mobile phones. Inspired by the loop perforation technique from source code optimization, we speed up the bottleneck convolutional layers by skipping their evaluation in some of the spatial positions. We propose and analyze several strategies of choosing these positions. We demonstrate that perforation can accelerate modern convolutional networks such as AlexNet and VGG-16 by a factor of 2× -4×. Additionally, we show that perforation is complementary to the recently proposed acceleration method of Zhang et al. [28].
The goal of this paper is to reduce the evaluation time and memory size during evaluation of a CNN. The paper proposes by saving time by only evaluating a subset of convolutional features at selected spatial locations, determined by a “perforation mask”. The non-evaluated spatial locations are then filled in through nearest-neighbor interpolation. The paper proposes a few methods for determining this perforation mask. The paper evaluates the performance of this technique on recent CNN architectures: Network-in-network, AlexNet, and VGG-16.
This paper proposes a method to speed up the evaluation of CNNs. The method skips the evaluation of a subset of spatial locations in the image, and interpolates their values using nearest neighbor values. Methods for accelerating CNNs are of great interest to the community at this time. The approach of the method is well-described. I did have a few concerns with some of the results and parts of the method.  (1) Figure 4(c)(d) shows the increase in error vs speedup factor for several methods. The impact line resulting in greater errors than all of the other methods in the conv3 layer is a surprising result, and some additional analysis would shed more light on the result to the reader.  It may also indicate that the impact is not being computed with the correct metric. A feature which is not computed will be interpolated, rather than zeroed out, before evaluation in the next layer. Why then, is “impact” measured by estimating the effect of zeroing out the feature (feature magnitude times gradient), rather than just the effect of perturbing the feature (simply the gradient)?  (2) The baselines all corrupt the CNN function. The “resize” operation changes the input scale of the image. The “stride” operation reduces the feature map resolution at the layer, so the subsequent layer is effectively performing an a trous/dilated convolution on the feature map at the original resolution, which means that it is evaluating a different function. I believe the true baseline is simply the perforated-CNN with zero-filling instead of interpolation.   (3) Though there is no pooling layer directly after conv3 in AlexNet, the “pooling structure” method for perforation mask selection could reasonably be extended to conv4 and conv3, since layers conv3-conv5 share the same spatial locations.  (4) For VGG, “pooling structure” will have a tie in all spatial locations. Since kernel&stride are equal in the network, no features are in overlapping regions. This should be mentioned, either in the page 3 line 123 or page 7 line 225.  (5) The barycentric interpolation is likely to be of interest to the reader. I would recommend moving this material from the appendix into the paper, with an analysis of its classification performance, memory, and evaluation time.