Solved by ptrblck in post #2 
              
              
                No, since model.eval() and model.train() do not change the gradient calculation or the backpropagation in general. Instead the behavior of certain layers is changed. E.g. dropout layers are disabled during eval() and batchnorm layers will use their internal running stats to normalize the input acti…