Hi,
I don’t think this cause a memory leak, but that can increase the peak memory usage.
Indeed the whole history corresponding to your loss is linked to the loss python variable. This used to be cleared when .backward() is called. But when it’s not called, it’s only cleared when the loss variable is overriden. Unfortunately, this only happens after the next forward pass. So the peak memory usage will be much larger.
You can add a del loss when you do not call .backward() to reduce the peak memory usage.