Solved by albanD in post #2 
              
              
                Hi, 
cudnn has many algorithms it can choose from to perform convolution. This choice depends on the input size and so a small change in input size might trigger a different algorithm to be used and thus different memory behavior. 
You can try setting torch.backends.cudnn.benchmark=True for cudnn to…