Hi,
Is there any way to measure the peak cuda memory usage without hurting the execution time?
I know there is multiple functions kindly provided in torch.cuda. But they don’t include the CUDA overhead memory. i.e. There is always a gap between torch.cuda.max_memory_reserved() and what I see in nvidia-smi.
An alternative solution would be initiating a thread to keep calling nvidia-smi and track the maximum value. But threading will hurt the execution time of the main process, which is also the other term I want to benchmark.
Is there any solution for measuring peak gpu memory usage and also total execution time?
Thanks very much.