I have already spent a lot of time on this problem. My criterion CrossEntropyLoss. The input is [batch, num_class], the target is [batch]. Everything works well on batch training and testing. On a single package does not work. I think that NN sees the data inside the batch. We have to train the model with the size of the batch = 1.