SimonW:


But I suppose there is no way pytorch could even try to estimate the extras that it can’t account for.

You are right. Unfortunately we can’t know the exact size of the cuda context. The memory_allocated/cached methods report all memory allocated by pytorch. But if cuda context is out of our control. But, I think you can safely assume that cuda context is 100~250MB.


Are you talking about the context of loading cuda, the first time it’s used? I get a pretty consistent 0.5GB per process with cuda100.
If you’re talking about cuda context allocated during the actual use of cuda, how can it be a relatively fixed number, regardless of memory usage? clearly, I know very little about it cuda context, and can only talk about the actual usage numbers I see. Do you have a recommendation for a concise description/guide to read to understand cuda context w/o a need to study something huge?
Thank you.