Are you randomly pruning? If so I would suspect that the pruning is considering all weights (including previously pruned) in the computation: If I randomly set 0.1 of the weights to 0 for 10 iterations, I expect (in lieu of LaTeX support in the forum) sum([0.1 * (0.9**i) for i in range(10)]) ~ 65.1% of weights to be set to 0.
This appears to be consistent with the documentation in Pruning Tutorial — PyTorch Tutorials 1.9.1+cu102 documentation . If you have some other criterion, maybe it is applied to the original weights. I have to admit that I don’t know how it exactly works from the top of my head.
Best regards
Thomas