These numbers naturally don’t match exactly as nvidia-smi because of cuda context.

Which renders the pytorch method memory_allocated() not so useful, because it doesn’t help to know how much memory was really allocated, if one is trying to calculate their hyperparameters based on available free-memory. But I suppose there is no way pytorch could even try to estimate the extras that it can’t account for. It sounds like something is wrong in that logic, but unfortunately I don’t know anything about CUDA internals to tell why it can’t communicate the client how much memory it allocated so that that memory_allocated could match the real thing. So I guess direct access to the the gpu via nmvl is really the only method that I know of that can be used for precise measurements, and pytorch’s memory_allocated is more of an estimate.

I agree that resetting methods will be useful. Please submit a feature request.

Done: https://github.com/pytorch/pytorch/issues/15968 8
Thank you for following up on this thread, @SimonW