Hello @ptrblck! Thanks for answering.
Yes, I think CUDA context takes around 900MB for me, but after interruption I get 3-4GB that I cannot release (by deleting all objects / exiting function scope).
I produce some simple exception (like ZeroDivision) and it doesn’t kill kernel (cause I can still access all my variables from other cells after error).
But when I then call torch_clear_gpu_mem(), CUDA memory reliably returns to 900MB.