Filter based Deep Compression with Global Average Pooling for Convolutional Networks
Deep neural networks are powerful, but using these networks is both memory and time consuming for their numerous parameters and large amounts of computation. There are many studies in compressing the models. On the parameter level, one of the compressing methods is to spend lots of time performing the iterative process consisting of pruning weights and fine tuning the models. On the bit level, many studies use quantization to cut down on the number of need bits. Hence, we propose an efficient strategy to compress on the layers which are computation or memory consuming. We compress the model by adding the global average pooling, iteratively pruning on the filters with proposed order deciding scheme to prune more efficiently, applying the truncated SVD to the fully connected layer, and performing the two stage quantization. Experiments on the VGG16 model show that we can reach a 60.9x compression ratio in off line storage with about 0.848% and 0.1378% loss of accuracy on the top 1 and top 5 classification results with the validation dataset of ILSVRC2012.
2018 IEEE International Workshop on Signal Processing Systems (SiPS)