albanD:

This operation on x is not valid. You cannot change its value in a differentiable manner as it is a leaf…
If you just want to zero-out the content of x without the autograd knowing, you should do:
with torch.no_grad():
  x.zero_()



I was reading your post on what torch.no_grad() does and it says I won’t be able to do backward passes on my computation graph if I use that (though with some speed/memory benefits).
That is not what I am trying to do. I want things to be differentiable but I want to be able to clear out gradients of sub-graphs whenever I want.
More clearly explained: I create a forward pass and in a earlier part of the graph I want to collect gradients to later use those gradients to create the whole computation graph. But the gradients are part of the graph and I want them zeroed out since they should be treated as constants part of the computation graph. Thus I will call backward twice and zero out the intermediate gradients collected (but I will extract to use them later).
Does that make sense?