Solved by ptrblck in post #2 
              
              
                The train() and eval() call change the internal self.training flag, so you could grep for it in the source folder (grep -r self.training). 
Currently it seems these modules are affected by it in the PyTorch core: 

Quantization modules
Dropout
InstanceNorm
BatchNorm
RNN (probably only if using cudnn…