Ah OK, thanks for the information.
I assume the validation loss could at least be detached.
Could you also post another figure showing the increase in memory usage?
The first posted image (with torch.cuda.empty_cache()) doesn’t show a clear increase.