Feature Map Transform Coding for Energy-Efficient CNN Inference
Convolutional neural networks (CNNs) achieve state-of-the-art accuracy in a variety of tasks in computer vision and beyond. One of the major obstacles hindering the ubiquitous use of CNNs for inference on low-power edge devices is their high computational complexity and memory bandwidth requirements. The latter often dominates the energy footprint on modern hardware. In this paper, we introduce a lossy transform coding approach, inspired by image and video compression, designed to reduce the memory bandwidth due to the storage of intermediate activation calculation results. Our method does not require fine-tuning the network weights and halves the data transfer volumes to the main memory by compressing feature maps, which are highly correlated, with variable length coding. Our method outperform previous approach in term of the number of bits per value with minor accuracy degradation on ResNet-34 and MobileNetV2. We analyze the performance of our approach on a variety of CNN architectures and demonstrate that FPGA implementation of ResNet-18 with our approach results in a reduction of around 40% in the memory energy footprint, compared to quantized network, with negligible impact on accuracy. When allowing accuracy degradation of up to 2%, the reduction of 60% is achieved. A reference implementation accompanies the paper.
A lossy transform coding approach was proposed to reduce the memory bandwidth of edge devices deploying CNNs. For this purpose, the proposed method compresses highly correlated feature maps using variable length coding. In the experimental analyses, the proposed method outperforms some of the previous work in terms of the compression ratio and accuracy for training ResNet-34 and MobileNetV2. 
The proposed method and initial results are promising. However, the paper and the work should be improved for a clear acceptance:
- Some parts of the method need to be explained more clearly:
– In the statement “Due to the choice of 1 × 1 × C blocks, the PCA transform essentially becomes a 1 × 1 tensor convolution kernel”, what do you mean by “the PCA transform becomes a convolution kernel.”?
- Could you please further explain how you compute PCA using batch data, how you update online and how you employ that in convolution weights together with BN? Please also explain the following in detail:
(I) “To avoid this, we calculate the covariance matrix layer by layer, gradually applying the quantization.” What is the quantization method you applied, and how did you apply it gradually?
(II) “The PCA matrix is calculated after quantization of the weights is performed, and is itself quantized to 8 bits.” How did you quantize the weights, how did you calculate PCA using quantized weights and how did you quantize them to 8 bits?
- Could you please explain the following settings, more precisely: direct quantization of the activations; quantization of PCA coefficients; direct quantization followed by VLC; and full encoder chain comprising PCA, quantization, and VLC? Please note that there are various methods and algorithms which can be used for these quantization steps. Therefore, please explain your proposed or employed quantization methods more clearly and precisely.
–  Please clarify the statement “This projection helps to concentrate most of the data in part of the channels, and then have the ability to write less data that layers.”.
- Did you apply your methods to larger networks such as larger ResNets, VGG like architectures, Inception etc?
- I also suggest you to perform experiments on different smaller and larger datasets, such as Cifar 10/100, face recognition datasets etc., to examine generalization of the proposed methods at least among different datasets.