title: Coarse Pruning of Convolutional Neural Networks with Random Masks
abstract: The learning capability of a neural network improves with increasing depth at
  higher computational costs. Wider layers with dense kernel connectivity patterns
  further increase this cost and may hinder real-time inference. We propose feature
  map and kernel level pruning for reducing the computational complexity of
  a deep convolutional neural network. Pruning feature maps reduces the width
  of a layer and hence does not need any sparse representation. Further, kernel
  pruning changes the dense connectivity pattern into a sparse one. Due to coarse
  nature, these pruning granularities can be exploited by GPUs and VLSI based
  implementations. We propose a simple strategy to choose the least adversarial
  pruning masks. The proposed approach is generic and can select good pruning
  masks for feature map, kernel and intra-kernel pruning. The pruning masks are
  generated randomly, and the best performing one is selected using the evaluation
  set. The sufficient number of random pruning masks to try depends on the pruning
  ratio, and is around 100 when 40% complexity reduction is needed. The pruned
  network is retrained to compensate for the loss in accuracy. We have extensively
  evaluated the proposed approach with the CIFAR-10, SVHN and MNIST datasets.
  Experiments with the CIFAR-10 dataset show that more than 85% sparsity can be
  induced in the convolution layers with less than 1% increase in the misclassification
  rate of the baseline network.
authors: Sajid Anwar Wonyong Sung
hIndex: Sajid Anwar 31