So we have an implementation of weights pruning on this repo 314. The idea is to create a wrapper on the linear or conv layer, and apply the mask on the forward pass. Since multiplying the mask is a differentiable operation (multiplying constant essentially), PyTorch AutoDiff will take care of the backward pass automatically.
Well still you may not expect any acceleration, the goal of this implementation is to study the properties of pruned network.