This paper propose a cumulative saliency based Globally Soft Filter Pruning (GSFP) scheme to prune redundant filters of Convolutional Neural Networks (CNNs). Specifically, the GSFP adopts a robust pruning method, which measures the global redundancy of the filter in the whole model by using the soft pruning strategy. In addition, in the model recovery process after pruning, we use the cumulative saliency strategy to improve the accuracy of pruning. GSFP has two advantages over previous works: (1) More accurate pruning guidance. For a pre-trained CNN model, the saliency of the filter varies with different input data. Therefore, accumulating the saliency of the filter over the entire data set can provide more accurate guidance for pruning. On the other hand, pruning from a global perspective is more accurate than local pruning. (2) More robust pruning strategy. We propose a reasonable normalization formula to prevent certain layers of filters in the network from being completely clipped due to excessive pruning rate. Experiment results show that GSFP is effective on many classic CNN architectures and different data sets. Within my knowledge, GSFP is the first algorithm to combine global and soft pruning strategies. Notably, on MNIST and CIFAR-10, it achieves a much higher compression ratio compared with prior work while maintaining the same test accuracy.
This paper proposes new heuristics to prune and compactify neural networks. The heuristics try to consider 1) filter weight and gradient normalization by their size, 2) saliency normalization across layers, 3) saliency accumulation across batch. The author claims that these can address problems previous studies had and experimental results show that the proposed method achieve higher compression ration with less loss of accuracy.
This paper discusses how to determine the importance of filters. As cited in the paper, there have been various attempts to tackle the same problem and the paper contributes to the series of efforts. The paper introduces a new way to compute such importance values based on their observations. The method is tested on a few dataset and a various models and compared with some previous studies. I like the simple but yet effective method, however, I think it is not good enough for ICLR. 
1. What is effective is not very clear.
The paper pointed out issues of previous studies and proposed the new method based on the observations. However, only the final method is compared with other work and it did not examine which part of the method was essential. The paper needs more detailed analyses on the proposed method. For example, the readers would want to know if the normalization in Eq. (2) is really important or not. The readers would be also interested in a visualization like Fig. 2 without saliency normalization. 
2. The numbers of previous studies come only from their papers.
It is very difficult to know if the proposed method is actually better than the previous methods if the numbers just come from their papers. We want to compare the ideas, but not numbers. The essential ideas of other papers need to be abstracted and tested in the paper by itself. It relates to the first item above. "Baseline" should be a baseline method but not models without pruning.
Numbers from other papers are still useful to show that the numbers in the paper are good in an absolute manner.
3. Weak theoretical reasoning
Eq. (1) in the paper is not actually used for optimization while some previous methods do. If the proposed method is better than other methods which directly optimizes the loss, should we think that the formulation itself is bad? 
The paper discusses imbalanced pruned pruning results. It needs to show that it is actually bad.
* minor things
** Table 1: Should the first row of "Wen et al. (2016)" have "5-19" and "1-5" or "4-19" and "1-4" for "Filters" and "Channels", respectively?
** I'd recommend another proofreading.