Thank you so much again for these precious tips. I just had another question on this topic. Is there a way to free the tensors saved for backwards or the grad_output before the end of backward?
Say I have something like:
def backward(cls, ctx, grad_output):
.
.
.
   del grad_output;
.
.
.

I imagine that the above is pointless since there would still be a reference for grad_output on the call stack, right? Would there be a way do to something like this?