Again, thank you for the quick reply. The code sample is for manually calculating the gradient layer by layer on the backward. I wonder if you have other better way to achieve this goal. I did lots of searches on how to enable this, but this is the only way that I found out. It would be greatly appreciated if you could give some possible hints.
In addition, you mentioned that tensors still be created to storage the gradients. I wonder if you mean that
(1) the outputs on user-defined tensor are actually copied from some internal result tensors,
(2) this also applies to the regular .backward() case (meaning that we don’t pre-assign tensor to .grad but let this being done automatically),
(3) this is not avoidable since we cannot directly write the results in-place.