Hi,
The Tensor.to() op is not  detaching and gradients will flow back to a.
When you do
result1 = net1(a.to(“cuda:0”))
result2 = net2(a.to(“cuda:1”))

Then all the gradients will flow back all the way to a properly.

Also, any API that I could use to check whether two tensors are tached?

I’m afraid there is not.
You can check by side effect by doing: autograd.grad(out, inp). This will try to compute the gradient and raise an error if inp is not attached to out.