darylnak:

I was told by someone that clone() will destroy the gradients of x (?)


It is not true. Check this example.
a = torch.tensor(1., requires_grad=True)
b = 2*a
b.backward()

a.grad
# 2.

a_ = a.clone()

a.grad
# 2. 

Also, I think you need to do a backward pass through the tensor that you cloned. So you should not use .detach() as that sets the requires_grad=False of the returned tensor.