albanD:

I would avoid using .item() in pytorch as it unpacks the content into a regular python number and thus it breaks gradient computation. If you want to have a new Tensor such that no gradients will flow back, you should use .detach() .


Hi AlbanD,
I was puzzled about your comment on using .item(). Why would it break gradient computations? Do you have a high-level explanation of why it does that?
Perhaps more importantly, doesn’t that mean that using python floats in torch might cause problems? (since that’s what .item() extracts) Does that mean I should wrap everything in torch variables?
Thanks in advance!