I noticed something strange, when you run:
a = torch.tensor(0)
b = torch.tensor(1)

print(id(a) == id(b))
print(id(a.detach()) == id(b.detach()))

The output of the first print is False whereas it’s True for the second - the detached tensors get the same python id (i.e. memory address). Is this behavior expected?
What’s worse, this seems to be nonteterministic:
for i in range(20):
    print(id(a.detach()) == id(b.detach()))

if you run this loop a few times in a row, you should notice that the detached tensors occasionally get different memory addresses. What’s going on here?