torch.cuda.max_memory_allocated() returns the peak memory usage so it would be interesting to hear more about your debugging and how you’ve tried to infer a memory leak from it.
E.g. what did memory_summary() return and did you see “lost” allocations?