Deep learning has been attracting enormous attention from academia as well as industry due to its great success in many artificial intelligence applications. As more applications are developed, the need for implementing a complex neural network model on an energy-limited edge device becomes more critical. To this end, this paper proposes a new optimization method to reduce the computation efforts of convolutional neural networks. The method takes advantage of the fact that some convolutional operations are actually wasteful since their outputs are pruned by the following activation or pooling layers. Basically, a convolutional filter conducts a series of multiply-accumulate (MAC) operations. We propose to set a checkpoint in the MAC process to determine whether a filter could terminate early based on the intermediate result. Furthermore, a fine-tuning process is conducted to recover the accuracy drop due to the applied checkpoints. The experimental results show that the proposed method can save approximately 50% MAC operations with less than 1% accuracy drop for CIFAR-10 example model and Network in Network on the CIFAR-10 and CIFAR-100 datasets. Additionally, compared with the state-ofthe-art method, the proposed method is more effective on the CIFAR-10 dataset and is competitive on the CIFAR-100 dataset.
This paper motivates itself by observing that not all convolutional weights are required to make an accurate prediction. In the introduction the authors envision a system similar to a cascaded classifier [Viola and Jones 2001] (I draw this conclusion not the paper). However the wording of the introduction is not clear or it does not align with what is presented in the paper.
The approach in the paper does not perform early stopping dynamically during the feedforward phase. The approach removes weights which do not impact the accuracy after training has completed and the fine tunes the resulting network.
The clarity of the introduction must be addressed however the work is still interesting. I recommend the authors try to make the introduction as accessible as possible.
Also there are very general statements like "The activation layer introduces non-linearity into the system for obtaining better accuracy." which do not contribute to the message of the paper. The paper will be fine without these statements. Shorter is better.
Section 3 is good as a motivating example. However the conclusion “Thus, our focus is to develop an effective method of choosing a good intermediate result for saving more MAC operations with less accuracy drop” is not very clear. More insights written hear would be better.
One major flaw is that no analysis with respect to time of computation was performed. GPUs offer the advantage of being optimized for convolutions so it is possible that there is no speedup. Because of this it is unclear if the method would save time. The results clearly show individual computations (MACs) are reduced but it is not clear how this correlates with wall clock time.
Why do you start with the centre layers? I understand the heuristic you’re using, that the middle layers won’t have high or low-level features, and that they won’t break the other layers as badly if you modify them, but I feel like this is core to your method and it’s not adequately justified. I’d like to see some experiments on that and whether it actually matters to the outcome. Also, you don’t say if you start at the middle then go up a layer, or down a layer. I think this matters since your proposed C10-Net has only 3 convolutional layers.
All the filters in the same layer share a common checkpoint. Is that a good idea? What is the cost of doing this on a per-filter level? What is the cost on a per-layer level? Discussing runtime estimates for the added computation at training would make it more clear what the cost of the method is.
In section 4.4.3. you mention that the majority of weight distributions in CNNs follow the Gaussian manner. Could you cite something of this? You might also want to move that in Step 1 (section 4.1.), since it seems to be your motivation for selection of checkpoint locations (5% and 32%) and I had no idea why you selected those values at that point.
Typos:
Typo on page 3: “exploits redundancies inter feature maps to prune filters and feature maps”
Structural:
Maybe section 4.3 should be part of the description of Section 4. Proposed Method, not its own subsection.
Maybe table 2 should go at the end of section 4.4.1, because you describe it the error and columns in section 4.4.1.