Solved by ptrblck in post #8 
              
              
                Yes, the .peak stats will give you the maximum. You can use torch.cuda.reset_peak_memory_stats() to reset this peak if you need to monitor another peak usage.