Hi,
Note that to detach a Tensor, you should not do: x = Variable(x.data, requires_grad=True) but x = x.detach().requires_grad_(). (.data has other side effects and should not be used anymore).
Same for the backward, you don’t need the .data.
You can pre-allocate the gradients like this, yes. But in the code sample you gave, it has very limited benefit (I assume because this is only part of your real code).
Note though that during the backward pass, Tensors will still be created to store the gradients of the intermediary ops).