Solved by russellizadi in post #2 
              
              
                When the clone method is used, torch allocates a new memory to the returned variable but using the detach method, the same memory address is used. 
Compare the following code: 
import torch
device = torch.device("cuda")
a = torch.randn([10000, 10000])
a = a.to(device)
print(round(torch.cuda.memory_a…