Is there a way to delete all .grad attributes after training?
I am currently implementing some pruning techniques that reduce the dimensions of the weight tensors in convolution layers. For this to work I need to set all gradients to None or delete them since they don’t match the size of the filters anymore. However even after setting all of the weight.grad to None I still receive an error:
File "/home/bullseye/Documents/git/Master-Thesis-Pruning-for-Object-Detection/docker-omgeving/Code/trainengine.py", line 39, in process_batch
    loss.backward()
  File "/home/bullseye/.local/lib/python3.6/site-packages/torch/tensor.py", line 166, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/bullseye/.local/lib/python3.6/site-packages/torch/autograd/__init__.py", line 99, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: Function CudnnConvolutionBackward returned an invalid gradient at index 1 - got [1024, 527, 3, 3] but expected shape compatible with [1024, 767, 3, 3]

So I am either missing some gradients or the ones I’m setting to None don’t act like I thought they would.