stas:

But I suppose there is no way pytorch could even try to estimate the extras that it can’t account for.


You are right. Unfortunately we can’t know the exact size of the cuda context. The memory_allocated/cached methods report all memory allocated by pytorch. But if cuda context is out of our control. But, I think you can safely assume that cuda context is 100~250MB.



 stas:

Done: https://github.com/pytorch/pytorch/issues/15968 


I submitted a patch to add those methods. Could you check the PR (linked in the issue) and let me know if they satisfy your needs?