I am unsure of how to provide a minimal code snippet of my program that is able to reproduce the error since it is several hundreds of lines. However I have found the solution to my problem.
As I first suspected it was the grad attributes of the tensors that were the problem. The size of these tensors were different to the size of the weights itself after pruning them. Setting them to None solved the issue.
The first time I attempted this solution I forgot that the running mean and running var tensors of batch normalization layers also contain gradients.
@ptrblck Thank you for your help, your dummy code helped me understand and find the problem better. I am however still unsure of why your dummy code doesn’t face the same issues.