@eqy   I was running a model inference and the batch size are both 1. The code on two GPUs are exactly the same. I just copied one to the other. And there’s no other program running or batch size adjustment. I found the memory usage depends on the GPU type. And I found torch.cuda.empty_cache() can sometimes get the minimum usage. But it sometimes doesn’t work.