Pruning neural network parameters is often viewed as a means to compress models, but pruning has also been motivated by the desire to prevent overfitting. This motivation is particularly relevant given the perhaps surprising observation that a wide variety of pruning approaches increase test accuracy despite sometimes massive reductions in parameter counts. To better understand this phenomenon, we analyze the behavior of pruning over the course of training, finding that pruning's effect on generalization relies more on the instability it generates (defined as the drops in test accuracy immediately following pruning) than on the final size of the pruned model. We demonstrate that even the pruning of unimportant parameters can lead to such instability, and show similarities between pruning and regularizing by injecting noise, suggesting a mechanism for pruning-based generalization improvements that is compatible with the strong generalization recently observed in over-parameterized networks.
This paper mainly studies the relationship between the generalization error and mean/variance of the test accuracy. The authors first propose a new score for pruning called E[BN]. Then, the authors observe the generalization error and the test accuracy mean/variance for pruning large score weights and small score weights for VGG11, ResNet18, and Conv4 models. From these experiments, the authors observe that pruning large score weights generates instable but high test accuracy and smaller generalization gap compared to pruning small score weights. The authors additionally study some other aspects of pruning (e.g., pruning as a noise injection) and conclude the paper.
Overall, I am not sure whether the observation holds in general due to the below reasons. 
- The authors proposed a new score E[BN] and all experiments are performed on this. However, how it differs from the usual magnitude-based pruning in practice is unclear. I would like to know whether similar behavior is observed for the naïve magnitude-based pruning.
- I think that the author’s observation is quite restricted and cannot extend to a general statement since the experiments are only done for pruning small score/large score weights. To verify the generalization and instability trade-off, I believe that it is necessary to examine several (artificial) pruning methods controlling the instability of test accuracies and check whether the proposed trade-off holds. For example, one can design pruning methods that disconnect (or almost disconnect) the network connection from the bottom to the top (i.e., pruned network always outputs constant) with some probability to extremely increase the instability.
- The authors did not report the results for high sparsity.
Besides, I am not sure the meaning of the instability since when the test accuracy of the pruned model is higher than that of the unpruned model, the instability could be large.
Other comments:
- The first paragraph mentions that the generalization gap might be a function of the number of parameters. However, I think that it is quite trivial that the generalization gap is not a function of the number of parameters while it only provides the upper bound.
----------------------------------------------------------
I have read the authors' response. Thanks for clarifying the definition of instability and additional experiments with high sparsity. However, I will maintain my score due to the following concern. 
The remaining concern is that the current evidence for verifying generalization-stability tradeoff is not convincing as the authors presented only some examples having small and large instability (e.g., pruning smallest/largest weights) under the same pruning algorithm. I think that the results would be more convincing if the authors add a test accuracy plots given a fixed prune ratio, whose x-axis is controlled instabilities (e.g., from 10% to 90%) among various pruning algorithms (other than magnitude-based ones, e.g., Hessian based methods). It would be much more interesting if the same instability results same test accuracy even for different pruning algorithms.