I’m having trouble finding my memory leak, and I’m trying your 3rd tip which is using the continue after each line and check. I have a small question about it: if we continue right after a forward call, should the memory consumption stay constant? Here is my code:
y_pred, y_est = model[model_id](x)
print(torch.cuda.memory_allocated() / torch.cuda.max_memory_allocated())
continue

The forward call is the first thing in the training loop, and the memory starts to explode. Is this expected or does this mean the leak is likely inside the call? Thank you.