Solved by albanD in post #5 
              
              
                If you need a new Tensor every time, then you will need to create it every time, there is not way around it. You can use torch.zeros(your_size, device="gpu") to create it directly on the gpu and avoid the copy. 
If you use it once then you can use resize_().zero_() (after detaching the current Tenso…