Neural network pruning techniques have demonstrated it is possible to remove the majority of weights in a network with surprisingly little degradation to top-1 test set accuracy. However, this measure of performance conceals significant differences in how different classes and images are impacted by pruning. We find that certain examples, which we term pruning identified exemplars (PIEs), and classes are systematically more impacted by the introduction of sparsity. Removing PIE images from the test-set greatly improves top-1 accuracy for both sparse and non-sparse models. These hard-to-generalize-to images tend to be mislabelled, of lower image quality, entail abstract representations, atypical examples or require fine-grained classification.
The paper claims that neural network pruning methods have different impact on accuracy in class-wise and sample-wise. To this end, the paper performs statistical tests to identify such classes (and samples) from a population of neural network models of different levels of pruning. In particular, the identified samples, called PIEs, are shown to have significantly lower test accuracy, i.e., the PIEs are much harder to classify, and a user study is performed to support this claim. The paper also demonstrates that pruned models tend to have lower accuracy against adversarial attacks or common corruptions.  
In overall, the paper addresses an important problem of investigating the effects of pruning. The experiments performed here seems fairly extensive. One of my primary concerns, however, is that I am still not convinced whether the empirical findings presented here is indeed significant, as some reader might feel those results are not that surprising. I personally feel that it is much more likely that pruning gives heterogeneous effects over the classes, since current pruning schemes do not enforce the "uniform" brain damage, i.e., they simply aim to minimize total accuracy drops. Also, the accuracy differences in Table 1 do not appear to be that large in my opinion. The questions in what follows may help to resolve such concerns: 
- Section 2.1: For ImageNet dataset, data imbalance in training set might be the reason of such disparate impact? As CIFAR-10 dataset is balanced, I wonder if CIFAR-10 results could be also presented in more details.
- What if the whole training and pruning is re-performed after excluding the classes which the accuracy is decreased more by pruning? Would it lead to better pruning results, or better overall accuracy?
- I feel there should be more explanation about the actual statistical implication of the use of Welch's t-test: Is it ok that S^c_t may not be independent to S^c_0? What is the key benefits of not just picking outlier classes among S^c_t with respect to c?
- What would happens if we exclude the PIEs in training set, and performs training & pruning from scratch? The overall claims may be strenghten if this could improve the pruning efficiency.
- Figure 7: I think the pure ImageNet top-1 accuracy should be also presented in each plot as a baseline for fair comparison, as this accuracy will be decreased with respect to the sparsity as well.