Hi,
Keep in mind that pytorch won’t release the memory after using it to make follow up allocation faster.
So this does not necessarily shows the overall needed memory but the maximum allocated memory.
Also you might want to check using torch.backends.cudnn.benchmark to allow it to find good kernels.
I am sure cudnn is heavily optimized for resnet’s convolutions