Hi,
The requires_grad_ call should be last of you do that in one line to make sure you get the leaf tensor.
Otherwise, the difference between all of these is going to be almost impossible to see !
By the way calling .cuda() on a cpu tensor will copy it, so no need to clone before. You can have a check and clone if it’s already on the correct device, otherwise just call .cuda(1) without the clone.