Please let me clarify. I don’t expect the loss of the classifier to decrease when evaluating, but I would expect it to be roughly the same as when it is being trained. The loss of the classifier during training is ~ 0.001 using nn.CrossEntropy but is ~ 4.1 when evaluating.