If you need a new Tensor every time, then you will need to create it every time, there is not way around it. You can use torch.zeros(your_size, device="gpu") to create it directly on the gpu and avoid the copy.
If you use it once then you can use resize_().zero_() (after detaching the current Tensor potentially). resize_() is more of an advanced function so maybe the first option above will be good enough for you.