title: On Iterative Neural Network Pruning, Reinitialization, and the Similarity of Masks
abstract: We examine how recently documented, fundamental phenomena in deep learn-ing models subject to pruning are affected by changes in the pruning procedure. Specifically, we analyze differences in the connectivity structure and learning dynamics  of  pruned models found through a set of common iterative pruning techniques, to address questions of uniqueness of  trainable, high-sparsity sub-networks, and their dependence on the chosen pruning method. In convolutional layers, we document the emergence of structure induced by magnitude-based un-structured pruning in conjunction with weight rewinding that resembles the effects of structured pruning. We also show empirical evidence that weight stability can be automatically achieved through apposite pruning techniques.
authors: Michela Paganini Jessica Forde
hIndex: Michela Paganini 48