@Soumith_Chintala I stumbled upon this thread and I observed that I just observed the opposite of this. The memory footprint for vgg16 pytorch model in my cases increases if I set cudnn.benchmark=False. For batchsize of 64, vgg16 pytorch version occupies 9978 MB of GPU memory. Is it normal or is there a way to reduce this ?