justusschock:

the other way around it is copied and then abandoned. Other than that, the methods should be identical.
Thus I’d recommend to first detach and then copy because it does not make sense to copy anything that is never used.


I am having trouble understanding this. I think the main issue I am having is because I do not understand .clone()'s semantics. The docs (https://pytorch.org/docs/stable/tensors.html#torch.Tensor 55) say:

Returns a copy of the  self  tensor. The copy has the same size and data type as  self .


NOTE: Unlike copy_(), this function is recorded in the computation graph. Gradients propagating to the cloned tensor will propagate to the original tensor.

I don’t understand what the point of recording the clone in the graph. Like what is the difference with doing:
import torch
import torch.nn as nn

a = torch.tensor([1,2,3.], requires_grad=True)
b = a # a copy of a, at least by reference in python
c = a.clone() # a copy of a by reference AND records operation in backward
d = torch.nn.Identity()(a) # same as c except it has different name for identity vs cloning

it seems to me that the only addition is that clone() records the operation in the graph, which seems really weird to. Why do we ever want to do this?
What is the difference between the above 4 operations? What is the use of clone? @albanD