pinocchio:

I was puzzled about your comment on using .item() . Why would it break gradient computations? Do you have a high-level explanation of why it does that?


Because it returns a python number and we cannot track gradients for python numbers, only torch Tensors.

Does that mean I should wrap everything in torch variables?

It will be done automatically for you whenever needed. The Tensors you create that require gradients are Tensor already.
So you don’t need to wrap anything. But you should not unwrap things either.

x.zero_()

This operation on x is not valid. You cannot change its value in a differentiable manner as it is a leaf…
If you just want to zero-out the content of x without the autograd knowing, you should do:
with torch.no_grad():
  x.zero_()