Since the msg board is full of pruning today… a while back, a researcher submitted a PR to one of my repos with some compelling pruned models (including a ResNet101 variant). They used a technique based on this paper: https://arxiv.org/abs/2002.08258 25
The pruned variants are created by applying an overlay in a txt file that defines the modifications to make to the original network, which was really easy to integrate. The end result is a network that is faster than the original and typically a model from the family one step down, while mainting a higher accuracy than the step down model. Unfortunately the code to generate he pruned models wasn’t included, but perhaps if someone asked they’d be willing to share.
See the original PRs: https://github.com/rwightman/pytorch-image-models/pull/132 26 and https://github.com/rwightman/pytorch-image-models/pull/136 13
Another researcher has recently reached out about integrating this pruning technique with my models: https://github.com/cmu-enyac/LeGR 36 … definitely some interesting progress in this area