I use same data when run with eval() and without it. Code look something like this. And my model trained to reproduce same value that it get as input.
model = create_model()
model.load_state_dict(torch.load('path_to_save'))
# model.eval() #<- uncomment for bad loss
loss_function = torch.nn.MSELoss()
logPdiff_var = Variable(torch.from_numpy(logPdiff)).type(dtype)
proxi = model.forward(logPdiff_var)
loss =  loss_function(proxi, logPdiff_var)
print(loss)

I try to run model with data and after this switch to eval() - doesn’t help
Without eval loss is 0.003 with it - 1.43 (this is loss at training start)
Small question: by default batchnorms in train mode? Or I should always call train() before training?