The thing is that you need somewhere to store the gradients. And in modern systems, adding into an existing buffer is much faster than allocating a new one then copying the values into it.
So this is not really a perf regression: re-using memory is actually a better way to do this.
If you really want to not do that, you can set the .grad fields to None instead of zeroing them out and a new Tensor to store the gradients will be set back every time.