ptrblck:

It doesn’t seem to make a difference, if I del the tensor and check the memory before and after this operation.


Yes, I totally agree.



 eduardo4jesus:

I imagine that the above is pointless since there would still be a reference for grad_output on the call stack


As far as I know, Python is heavily based on references instead of pointers and it manages memory allocation on its own. But, I wonder if there would be away to do something inside backward that would invalidate the reference in the call stack, or somehow just forcing a memory deallocation. I am just realizing now this would be a OFF-(Torch)-Topic question.