What is the difference between network sparsification and model pruning What is the difference between network sparsification and model pruning? I watched USENIX ATC '21 - Octo: INT8 Training with Loss-aware Compensation and Backward Quantization for Tiny (at 01:29sec) where they state them as two different methods to simplify the model.
I have difficulties understanding what is the difference.

There is no operational difference. A pruned network is usually leads to a sparser network. However, the degree of sparsification, i.e., how much it is pruned, would determine if it is qualified as "sparse network". One could think of difference between a dense matrix and a sparse matrix, from weight matrices perspective. Probably, rule of thumb is maybe if 90% or more connections are pruned, it is a sparse network compare to the original one, see The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks.
