I see torch.autograd.grad() can calculate the gradient for input and output, and returns the gradient. If I understand the code correctly, the returned gradient tensor is allocated while performing the computation. I wonder if it is possible to ask torch.autograd.grad() to output the results to a user predefined tensor.
or, I wonder if there is any other way to allow parameter.grad pre-allocated before the backward() operation.
My use case might be different from the common cases. I hacked into Pytorch’s memory management code to allow an allocated tensor to change its storage memory to a user-defined GPU memory. I also want to apply this on the gradient tensor. However, as gradient tensor is allocated during the backward() runtime, this makes my case very hard to realized on the gradient tensor as I don’t want to have an additional memory copy.