I did a very simple experiment today, codes are as follows:
with torch.no_grad():
print(torch.cuda.memory_reserved())
print(torch.cuda.memory_allocated())
print(os.system(‘nvidia-smi’))
inputs = torch.randn(1, 3, 512, 512).cuda()
print(os.system(‘nvidia-smi’))
conv = torch.nn.Conv2d(3, 64, (7, 7), stride=2, padding=3, bias=False).cuda()
inputs = conv(inputs)
print(torch.cuda.memory_reserved())
print(torch.cuda.memory_allocated())
torch.cuda.empty_cache()
print(torch.cuda.memory_reserved())
print(torch.cuda.memory_allocated())
outputs are as follows(remove some unused line):
0 ---- print(torch.cuda.memory_reserved())
0 ---- torch.cuda.memory_allocated()
487MiB /  8192MiB ---- print(os.system(‘nvidia-smi’))
before “torch.randn(1, 3, 512, 512).cuda()”, at this point maybe torch is not init and same as cuda, so this 487MiB is all windows used i think.
938MiB /  8192MiB ---- print(os.system(‘nvidia-smi’))
after “torch.randn(1, 3, 512, 512).cuda()”, at this point torch is init and “inputs” only consume little GPU memory, so about 500MB(944-487) is cuda context and so far everything can be explained
23068672 ---- print(torch.cuda.memory_reserved())
17863680 ---- print(torch.cuda.memory_allocated())
after inputs = conv(inputs), only one conv consume about 2G GPU memory, is this normal? and after the conv I think it should return to 938MiB, so I have thought there is memory leak in Pytorch.
23068672 ---- print(torch.cuda.memory_reserved())
17863680 ---- print(torch.cuda.memory_allocated())
after torch.cuda.empty_cache(), There is nothing happen, so I don’t understand why pytorch hold so many GPU memory